The issues identified in Analysis: Schema Compiler Backpointers/Copying require some design and test planning.

IsHidden

Implementation

This flag will be stored member is set on infoset elements at the time they are created (parsing) or spliced into the infoset (unparsing - streaming unparser). This requires using a stack at runtime that keeps track of what Terms, specifically what sequence group references are in the dynamic context, and if so, whether that sequence group reference is a hidden-group reference.

The Unparser, specifically InfosetInputter contains a "TRD stack" which is a stack of the TermRuntimeData objects for every Term in dynamic context.

Design alternatives

This works by dynamically maintaining boolean state member isInsideHiddenContext.

1. ParseOrUnparseState has member.

   Code Block
   var isInsideHiddenContext : Boolean = false

   
   

2. SequenceRuntimeData has constructor argument/member:

   Code Block
   val maybeHiddenGroupRefArg : Maybe[ModelGroupRuntimeData] lazy val maybeHiddenGroupRef = maybeHiddenGroupRefArg // and also serialized

   with value Nope for anything except a SequenceGroupRef that has the dfdl:hiddenGroupRef property. This is a Maybe type to provide for access to the referenced group's model-group runtime data. However, for purposes of isHidden computation this is just used as a boolean flag.

3. In the parse1 and unparse1 methods of Parser and Unparser, If
   1. the TRD of the current processor is of type SequenceRuntimeData, and
   2. trd.maybeHiddenGroupRef.isDefined, and
   3. the state.isInsideHiddenContext is currently false
4. TRD Stack
   1. Move the TRD Stack from the InfosetInputter so it is a member of the ParseOrUnparseState base class, where it is visible for both parsing and unparsing
   2. Update parsers to also maintain this stack.
   3. Whenever an infoset element is created, scan the stack to see if any Sequence Group Ref that is a hidden group ref, is on stack. If so mark the infoset element as hidden.
5. Maintain State member
   1. Add an isHiddenContext var member to ParseOrUnparseState
   2. In the parse1 => parse and unparse1 => unparse calls, examine the TRD of the current parser.
   3. If the TRD of the current processor is a hidden group ref, and the isHiddenContext is false, then invoke the parse/unparse method within a code branch that will set the isHiddenContext first sets the isInsideHiddenContext to true, and set sets it back to false upon unwinding. A Note that a try-finally type of unwind-protect could be used, but should not be necessary, as parsers are supposed to unwind the stack without throws, and all throws are fatal when unparsing.
   4. Otherwise, else invoke the parse/unparse method normally, which does not affect the state of isHiddenContext.

Tradeoffs:

• The TRD Stack incurs push/pop overhead only for Term processors, not for the primitive processors of delimiters, alignment regions, etc.
• The state member incurs a small overhead for every processor to determine if it is a hidden-group ref.
• The state member is more centralized solution in the code base, requiring only the member in the state, and code in the Parser and Unparser bases/traits in the parse1/unparse1 methods.
• The TRD Stack may be necessary for other things - it is needed for Unparsing at least in order for InfosetInputters to be able to properly construct element events.

1. 1. isInsideHiddenContext, so if already true, it stays true, if currently false, stays false.
2. When infoset elements are created by element combinators, (parsing), or when they are accepted and spliced into the infoset by element combinators (unparsing) they are setIsHidden(state.isInsideHiddenContext)

Test Plan/Design-for-Test

Existing tests that expect to determine isHidden from ERDs of elements should be removed/rewritten, as the information is no longer available there.

Tests should insure that elements are properly hidden if they are multiple group references away from a true dfdl:hiddenGroupRefResolved: Prefer maintaining a state member isHiddenContext and then setting a member of infoset elements when constructed.