A prior version of the proposal can be found here.

The Java org.xml.sax SAX API is a well known and understood API for handling XML data in an event driven manner. Beause the SAX API is event based, rather than other approaches such as DOM, it allows for efficient processing of XML with reduced memory usage.

This proposes changes Daffodil  to add support parsing and unparsing in conformance with the SAX API.

Note that this proposal does not address issues that are related to reducing Daffodil memory usage (a common benefit of the SAX API), such as creating InfosetOuputter events as early as possible rather than at the end of a parse, or allowing parts of the internal infoset representation to be garbage collected. Those issues will be resolved in separate proposalsmemory usage.

Requirements

• Daffodil shall implement the Java SAX API in accordance with the org.xml.sax SAX API documentation
• Daffodil shall maintain support for existing APIs
• Daffodil shall create SAX events representing a DFDL infoset while parsing data
• Daffodil shall receive SAX events representing a DFDL Infoset to unparse data

SAX Overview

The two main components of a SAX API are the XMLReader and ContentHandler interfaces.

...

Fortunately, these two SAX API components translate quite nicely to Daffodil's concepts of a Parser and Unparser. Much like a XMLReader, a Parser reads in data and converts it to XML. And much like a ContentHandler, an Unparser reads in XML and takes actions (i.e. unparses data) based on the XML. The following sections describe how Daffodil can implement the implements XMLReader and ContentHandler to support parse and unparse.

Parse

XMLReader

The XMLReader interface requires various getters and setters for internal mutable state. For example, the ContentHandler is set with the setContentHandler(...) method. Although it might feel natural to modify the DataProccesor to implement the XMLReader interface, the DataProcessor is considered to be immutable, which conflicts with the mutability of the XMLReader. For this reason, a new function is added to the DataProcessor to create an implementation of the XMLReader called a DaffodilParseXMLReader, for example:

...

In addition the the above functions, the following functinons are added to support functions support other input types that Daffodil supports, which may allow for some optimizations.

SAXInfosetOutputter

The SAXInfosetOutputter is an implementation of the Daffodil InfosetOutputter interface responsible for converting InfosetOutputter events to SAX ContentHandler events. According to the SAX API, applications may register a new or different ContentHandler with the XMLReader in the middle of a parse, and the SAX parser must begin using the new handler immediately. Because of this, the SAXInfosetInputter SAXInfosetOutputter must take the XMLReader as a parameter, and any time a SAX event is generated, it must call getContentHandler()  on that parameter. The definition for this class looks like:

...

Fortunately, the InfosetOutputter events correlate nicely to the InfosetOutputter ContentHandler events. Below is their mapping. Note that in some cases a single InfosetOutputter event may require calling multiple ContentHandler events.

...

Other functions in the ContentHandler interface will are not be usedused used.

Unparse

ContentHandler

The ContentHandler interface is used to receive and react to SAX XML events. In order to unparse data based on these events, Daffodil must unparse data based on the events that are received. However, the design of the unparser and InfosetInputters behaves opposite to this–rather than receiving events, the unparser and InfosetInputter requests the next event. This is essentially push vs pull, or SAX vs StAX. To support unparsing based on SAX events, we must convert these push-style SAX events into the pull style events that Daffodil requires.

...

How and when this event is mutated requires coordination between the push-style ContentHandler and the pull-style SAXInfosetInputter. This coordination is handled using coroutines and is describe in the following section.

Coroutine Coordination

The legacy Daffodil Coroutine library allows for pausing the execution of a subroutine to temporarily yield to the caller, and allow the caller to resume the coroutine back to where it paused earlier. This library was revived to coordinate the interactions between the ContentHandler and the SAXInfosetInputter.

...

class DaffodilUnparseContentHandler(dp: DataProcessor, output: OutputStream) extends ContentHandler with coroutine[SAXInfosetEvent] {
  
  private val infoseEvent = new SAXInfosetEvent()
  private val inputter = new SAXInfosetInputter(this, dp, output)


  private def sendtoInputter(....) {
    // queueing the infosetEvent for SAXInfosetInputter
	val infosetEventWithResponse = this.resume(inputter, Try(infosetEvent))
    infosetEvent.clear()
    // if event is wrapped in a Try failure, we will not have an unparseResult, so we only set
    // unparseResults for events wrapped in Try Success, including those events that have expected
    // errors
    if (infosetEventWithResponse.isSuccess && infosetEventWithResponse.get.unparseResult.isDefined) {
      unparseResult = infosetEventWithResponse.get.unparseResult.get
    }
    // the exception from events wrapped in Try failures and events wrapped in Try Successes
    // (with an unparse error state i.e unparseResult.isError) are collected and thrown to stop
    // the execution of the xmlReader
    if (infosetEventWithResponse.isFailure || infosetEventWithResponse.get.isError) {
      val causeError = if(infosetEventWithResponse.isFailure) {
        infosetEventWithResponse.failed.get
      } else {
        infosetEventWithResponse.get.causeError.get
      }
      causeError match {
        case unparseError: DaffodilUnparseErrorSAXException =>
          // although this is an expected error, we need to throw it so we can stop the xmlReader
          // parse and this thread
          throw unparseError
        case unhandled: DaffodilUnhandledSAXException => throw unhandled
        case unknown => throw new DaffodilUnhandledSAXException("Unknown exception: ", new Exception(unknown))
      }
    }
  }

  def startDocument() {
    // Start the coroutine
    infosetEvent.eventType = One(StartDocument)
    sendToInputter()
  }

  def endDocument() {
    infosetEvent.eventType = One(EndDocument)
    sendToInputter()
  }
  
  ...
}

class SAXInfosetInputter(unparseContentHandler: DaffodilUnparseContentHandler, dp: DataProcessor, output: OutputStream) extends InfosetInputter with coroutine[SAXInfosetEvent] {

  val currentEvent: SAXInfosetEvent = new SAXInfosetEvent
  val nextEvent: SAXInfosetEvent = new SAXInfosetEvent  

  def getEventType: InfosetInputterEventType = currentEvent.eventType.orNull
  def getLocalName: String = currentEvent.localName.orNull
  def getNamespaceURI: String = currentEvent.namespaceURI.orNull
  ...

  def hasNext: Boolean = {
    if (endDocumentReceived) false
    else if (!nextEvent.isEmpty) true
    else {
      val event = this.resume(unparseContentHandler, Try(currentEvent))
      copyEvent(source = event, dest = nextEvent)
      true
    }
  }

  def next(): Unit = {
    if (hasNext()) {
      copyEvent(source = Try(nextEvent), dest = currentEvent)
      nextEvent.clear()
      if (currentEvent.eventType.contains(EndDocument)) endDocumentReceived = true
    } else {
      // we should never call next() if hasNext() is false
      Assert.abort()
    }
  }
  ...
}

class SAXInfosetEvent() {

  var localName: Maybe[String] = Nope
  var simpleText: Maybe[String] = Nope
  var namespaceURI: Maybe[String] = Nope
  var eventType: Maybe[InfosetInputterEventType] = Nope
  var nilValue: Maybe[String] = Nope
  var causeError: Maybe[SAXException] = Nope
  var unparseResult: Maybe[UnparseResult] = Nope

  def isError: Boolean
  def clear: Unit
  def isEmpty: Boolean
}

...