Principles of Operation

Many global group defs exist with the intention that they are ONLY going to be used as hidden groups. An example of this is presence-bit indicator flags. These are 1-bit elements that live in a hidden group because they indicate the presence or absence of an element in the data. These flags can be used via dfdl:occursCount and dfdl:occursCountKind='expression', or via flags, choices, and discriminators. Either way they are a common case of hidden groups.

Other global group defs are created with the intention that they are only going to be used as non-hidden groups. However, users are free to use these as hidden groups.

Polymorphic Terms which are sometimes hidden, sometimes not, in the same schema, are expected to be far less common. Many schemas are expected to have no such groups.

Choices, and the choice branch maps used for unparsing, are related to the isHidden problem due to the transition plan below.

Runtime Checking

When an element is setIsHidden(true), then if it is of simple type it should be checked to insure it is either defaultable or dfdl:outputValueCalc, and it is a runtime SDE if not as shown above.

Implementation: Choice Combinators

ChoiceCombinator.unparser method computes a choiceBranchEventMap, and also determines statically which branch should be taken if the choice is hidden.

if trd.optIsKnownHidden is None, then both these are parameters to the ChoiceUnparserCombinator which decides at runtime if it is unparsing an isHidden choice, and selects to use the statically determined branch if so, otherwise if not hidden uses the choiceBranchEventMap. If trd.isKnownHidden is Some(true), then the ChoiceBranchEventMap can be omitted, and if trd.isKnownHidden is Some(false), then the statically determined branch can be omitted.

The ChoiceBranchEventMap is computed without regard for isHidden, that is, it assumes the current choice is not, itself hidden. Contents of each branch of the choice may contain hidden content or not.

Test Plan/Design-for-Test

Tests should insure that elements are properly hidden if they are multiple group references away from a true dfdl:hiddenGroupRef.

Test schemas with groups that appear both hidden and non-hidden are required to insure the runtime determination is exercised.

Transition Plan

Releases 2.5.0 and prior did not use this technique.

Existing tests that expect to determine isHidden from ERDs of elements should be removed/rewritten, as the information is no longer available there.

Existing code that expects to find isHidden on ERDs should get it from the Infoset Element (DIElement class) directly instead.

Since we cannot know if an element will be hidden or not until runtime, many checks currently done at schema compile time must also be done at runtime. For example, in childrenInHiddenGroupNotDefaultableOrOVC, a check is done for whether an element inside a hidden group is neither defaultable nor has dfdl:outputValueCalc. This test must be done at runtime as described above, in addition to doing it at compile time when optIsKnownHidden is Some(true).

Static isHidden parameters - all these must be removed. Anything that statically depends on knowledge of isHidden must be revised to not depend on full knowledge of it.

Transitioning to new IsHidden technique requires also transitioning the ChoiceBranchMap technique simultaneously.

• Static isHidden attribute is used by possibleFirstChildElementInInfoset.
• possibleFirstChildElementInInfoset is used by possibleNextChildElementInInfoset
• possibleNextChildElementInInfoset is used by nextParentElements which recursively uses possibleNextChildElementInInfoset
• possibleNextChildElementInInfoset is used by identifyingEventsForChoiceBranch.

Those are the only uses of the static isHidden attribute. While we still must compute identifyingEventsForChoiceBranch, we must do so without reference to static isHidden information. The members possibleFirstChildElementInInfoset, possibleLastChildElementInInfoset, and nextParentElements computations may be eliminated or adapted.

The algorithm for identifyingEventsForChoiceBranch must be adapted to not require upward navigation (back-pointers).

All such properties should be computed on global group definitions, not repeatedly for every group reference.

Algorithm TBD

• possibleFirstChildElementInInfoset calls possibleFirstChildTerms
• possibleFirstChildTerms calls possibleNextSiblingTerms
• possibleNextSiblingTerms calls enclosingTerms (note terms plural)

This seems problematic, as it is determining possible next sibling terms for ALL contexts. This may be problematic, and even allow erroneous/invalid infoset event streams to be unparsed.

We may need a runtime-structure akin to or perhaps the same as the TRD Stack technique used in InfosetInputter.