Standard Generalized Markup Language

SGML is an ISO standard: "ISO 8879:1986 Information processing – Text and office systems – Standard Generalized Markup Language (SGML)", of which there are three versions:

• Original SGML, which was accepted in October 1986, followed by a minor Technical Corrigendum.
• SGML (ENR), in 1996, resulted from a Technical Corrigendum to add extended naming rules allowing arbitrary-language and -script markup.
• SGML (ENR+WWW or WebSGML), in 1998, resulted from a Technical Corrigendum to better support XML and WWW requirements.

SGML is part of a trio of enabling ISO standards for electronic documents developed by ISO/IEC JTC1/SC34[1][2] (ISO/IEC Joint Technical Committee 1, Subcommittee 34 – Document description and processing languages) :

• SGML (ISO 8879) – Generalized markup language
  • SGML was reworked in 1998 into XML, a successful profile of SGML. Full SGML is rarely found or used in new projects.
• DSSSL (ISO/IEC 10179) – Document processing and styling language based on Scheme.
  • DSSSL was reworked into W3C XSLT and XSL-FO which use an XML syntax. Nowadays, DSSSL is rarely used in new projects apart from Linux documentation.
• HyTime – Generalized hypertext and scheduling.[3]
  • HyTime was partially reworked into W3C XLink. HyTime is rarely used in new projects.

SGML is supported by various technical reports, in particular

• ISO/IEC TR 9573 – Information processing – SGML support facilities – Techniques for using SGML[4]
  • Part 13: Public entity sets for mathematics and science
    • In 2007, the W3C MathML working group agreed to assume the maintenance of these entity sets.

SGML descended from IBM's Generalized Markup Language (GML), which Charles Goldfarb, Edward Mosher, and Raymond Lorie developed in the 1960s. Goldfarb, editor of the international standard, coined the "GML" term using their surname initials.[5] Goldfarb also wrote the definitive work on SGML syntax in "The SGML Handbook".[6] The syntax of SGML is closer to the COCOA format. As a document markup language, SGML was originally designed to enable the sharing of machine-readable large-project documents in government, law, and industry. Many such documents must remain readable for several decades—a long time in the information technology field. SGML also was extensively applied by the military, and the aerospace, technical reference, and industrial publishing industries. The advent of the XML profile has made SGML suitable for widespread application for small-scale, general-purpose use.

A fragment of the Oxford English Dictionary (1985), showing SGML markup

SGML (ENR+WWW) defines two kinds of validity. According to the revised Terms and Definitions of ISO 8879 (from the public draft[7]):

A conforming SGML document must be either a type-valid SGML document, a tag-valid SGML document, or both. Note: A user may wish to enforce additional constraints on a document, such as whether a document instance is integrally-stored or free of entity references.

A type-valid SGML document is defined by the standard as

An SGML document in which, for each document instance, there is an associated document type declaration (DTD) to whose DTD that instance conforms.

A tag-valid SGML document is defined by the standard as

An SGML document, all of whose document instances are fully tagged. There need not be a document type declaration associated with any of the instances. Note: If there is a document type declaration, the instance can be parsed with or without reference to it.

An SGML document may have three parts:

1. the SGML Declaration,
2. the Prologue, containing a DOCTYPE declaration with the various markup declarations that together make a Document Type Definition (DTD), and
3. the instance itself, containing one top-most element and its contents.

An SGML document may be composed from many entities (discrete pieces of text). In SGML, the entities and element types used in the document may be specified with a DTD, the different character sets, features, delimiter sets, and keywords are specified in the SGML Declaration to create the concrete syntax of the document.

Although full SGML allows implicit markup and some other kinds of tags, the XML specification (s4.3.1) states:

Each XML document has both a logical and a physical structure. Physically, the document is composed of units called entities. An entity may refer to other entities to cause their inclusion in the document. A document begins in a "root" or document entity. Logically, the document is composed of declarations, elements, comments, character references, and processing instructions, all of which are indicated in the document by explicit markup.

For introductory information on a basic, modern SGML syntax, see XML. The following material concentrates on features not in XML and is not a comprehensive summary of SGML syntax.

<QUOTE TYPE="example">
  typically something like <ITALICS>this</ITALICS>
</QUOTE>

<!ELEMENT chapter - - (title, section+)>
<!ELEMENT title o o (#PCDATA)>
<!ELEMENT section - - (title, subsection+)>

<chapter>Introduction to SGML
<section>The SGML Declaration
<subsection>
...

<chapter><title>Introduction to SGML</title>
<section><title>The SGML Declaration</title>
<subsection>
...

<!ELEMENT image - o EMPTY>

<QUOTE TYPE=example>
  typically something like <ITALICS>this</>
</QUOTE>

<QUOTE></QUOTE>

<QUOTE// <!-- not a typo! -->

<!ELEMENT lines (line*)>
<!ELEMENT line O - (#PCDATA)>
<!ENTITY   line-tagc  "</line>">
<!SHORTREF one-line "&#RE;&#RS;" line-tagc>
<!USEMAP   one-line line>

<lines>
first line
second line
</lines>

<lines>
<line>first line</line>
<line>second line</line>
</lines>

SGML has many features that defied convenient description with the popular formal automata theory and the contemporary parser technology of the 1980s and the 1990s. The standard warns in Annex H:

The SGML model group notation was deliberately designed to resemble the regular expression notation of automata theory, because automata theory provides a theoretical foundation for some aspects of the notion of conformance to a content model. No assumption should be made about the general applicability of automata to content models.

A report on an early implementation of a parser for basic SGML, the Amsterdam SGML Parser,[9] notes

the DTD-grammar in SGML must conform to a notion of unambiguity which closely resembles the LL(1) conditions

and specifies various differences.

There appears to be no definitive classification of full SGML against a known class of formal grammar. Plausible classes may include tree-adjoining grammars and adaptive grammars.

XML is described as being generally parsable like a two-level grammar for non-validated XML and a Conway-style pipeline of coroutines (lexer, parser, validator) for valid XML.[10] The SGML productions in the ISO standard are reported to be LL(3) or LL(4).[11] XML-class subsets are reported to be expressible using a W-grammar.[12] According to one paper,[13] and probably considered at an information set or parse tree level rather than a character or delimiter level:

The class of documents that conform to a given SGML document grammar forms an LL(1) language. ... The SGML document grammars by themselves are, however, not LL(1) grammars.

The SGML standard does not define SGML with formal data structures, such as parse trees; however, an SGML document is constructed of a rooted directed acyclic graph (RDAG) of physical storage units known as "entities", which is parsed into a RDAG of structural units known as "elements". The physical graph is loosely characterized as an entity tree, but entities might appear multiple times. Moreover, the structure graph is also loosely characterized as an element tree, but the ID/IDREF markup allows arbitrary arcs.

The results of parsing can also be understood as a data tree in different notations; where the document is the root node, and entities in other notations (text, graphics) are child nodes. SGML provides apparatus for linking to and annotating external non-SGML entities.

The SGML standard describes it in terms of maps and recognition modes (s9.6.1). Each entity, and each element, can have an associated notation or declared content type, which determines the kinds of references and tags which will be recognized in that entity and element. Also, each element can have an associated delimiter map (and short reference map), which determines which characters are treated as delimiters in context. The SGML standard characterizes parsing as a state machine switching between recognition modes. During parsing, there is a stack of maps that configure the scanner, while the tokenizer relates to the recognition modes.

Parsing involves traversing the dynamically-retrieved entity graph, finding/implying tags and the element structure, and validating those tags against the grammar. An unusual aspect of SGML is that the grammar (DTD) is used both passively — to recognize lexical structures, and actively — to generate missing structures and tags that the DTD has declared optional. End- and start- tags can be omitted, because they can be inferred. Loosely, a series of tags can be omitted only if there is a single, possible path in the grammar to imply them. It was this active use of grammars that made concrete SGML parsing difficult to formally characterize.

SGML uses the term validation for both recognition and generation. XML does not use the grammar (DTD) to change delimiter maps or to inform the parse modes, and does not allow tag omission; consequently, XML validation of elements is not active in the sense that SGML validation is active. SGML without a DTD (e.g. simple XML), is a grammar or a language; SGML with a DTD is a metalanguage. SGML with an SGML declaration is, perhaps, a meta-metalanguage, since it is a metalanguage whose declaration mechanism is a metalanguage.

SGML has an abstract syntax implemented by many possible concrete syntaxes; however, this is not the same usage as in an abstract syntax tree and as in a concrete syntax tree. In the SGML usage, a concrete syntax is a set of specific delimiters, while the abstract syntax is the set of names for the delimiters. The XML Infoset corresponds more to the programming language notion of abstract syntax introduced by John McCarthy.

Document markup languages defined using SGML are called "applications" by the standard; many pre-XML SGML applications were proprietary property of the organizations which developed them, and thus unavailable in the World Wide Web. The following list is of pre-XML SGML applications.

• Text Encoding Initiative (TEI) is an academic consortium that designs, maintains, and develops technical standards for digital-format textual representation applications.
• DocBook is a markup language originally created as an SGML application, designed for authoring technical documentation; DocBook currently is an XML application.
• CALS (Continuous Acquisition and Life-cycle Support) is a US Department of Defense (DoD) initiative for electronically capturing military documents and for linking related data and information.
• HyTime defines a set of hypertext-oriented element types that allow SGML document authors to build hypertext and multimedia presentations.
• EDGAR (Electronic Data-Gathering, Analysis, and Retrieval) system effects automated collection, validation, indexing, acceptance, and forwarding of submissions, by companies and others, who are legally required to file data and information forms with the US Securities and Exchange Commission (SEC).
• LinuxDoc. Documentation for Linux packages has used the LinuxDoc SGML DTD and Docbook XML DTD.
• AAP DTD is a document type definition for scientific documents, defined by the Association of American Publishers.
• ISO 12083, a successor to AAP DTP, is an international SGML standard for document interchange between authors and publishers.
• SGMLguid was an early SGML document type definition created, developed and used at CERN.

Significant open-source implementations of SGML have included:

• ASP-SGML
• ARC-SGML, by Standard Generalized Markup Language Users', 1991, C language
• SGMLS, by James Clark, 1993, C language
• Project YAO, by Yuan-ze Institute of Technology, Taiwan, with Charles Goldfarb, 1994, object
• SP by James Clark, C++ language

SP and Jade, the associated DSSSL processors, are maintained by the OpenJade project, and are common parts of Linux distributions. A general archive of SGML software and materials resides at SUNET. The original HTML parser class, in Sun System's implementation of Java, is a limited-features SGML parser, using SGML terminology and concepts.

• Overview of SGML Resources at W3C's website.
• Introduction and Examples of Software Documentation in SGML
• SC34 Committee Records, Charles Babbage Institute – Collection on the development of SGML and other standards influential in the development of current XML tools; documents include early drafts of SGML administrative materials, documentation, working group papers, and standards for computer languages.
• SGML Syntax Summary by Charles Goldfarb
• SGML document introducing you to SGML; Some reasons why SGML is important
• The SGML Declaration, in SGML and HTML Explained, Martin Bryan (1997) (the original URL is broken at http://www.is-thought.co.uk/book/sgml-4.htm#Fig4-2)
• SGML Declarations Wayne Wohler, IBM Corporation, 1994.
• ISO 9069:1988 – Information processing – SGML support facilities – SGML Document Interchange Format (SDIF)
• ISO/IEC 9070:1991 – Information technology – SGML support facilities – Registration procedures for public text owner identifiers