...

As part of the work for

You can see this in PR: https://github.com/apache/daffodil/pull/643

There is an IPv4 These include an IPv4 checksum layer example/test, and   a checkDigits example/test, and AIS Payload Armoring example/test. 

This illustrates These illustrate a way of doing "field extensibility" of layer transforms now, without any particular dynamic layer loading feature.

...

dfdlx:layerTransform='{ "someLayer" }

which because it is an expression it bypasses the validation checks against the LayerTransformEnum defined in the dfdlx.xsd schema. (in daffodil-propgen)

Then somewhere in Scala/Java code before compiling the schema one must call LayerTransformFactoryLayerCompilerRegistry.register(SomeLayerTransformFactorySomeLayerCompiler), and the SomeLayerTransformFactory SomeLayerCompiler object has to be defined on the classpath.

In the case of my the tests in daffodil-test, the scala test driver code does this call.

This is likely sufficient for application developers using Daffodil via API from software.

However: It won't let you a user run a new layer transform from the Daffodil CLI tool.

Enabling CLI Loading of Layers

To make the CLI work:

• The enum check in dfdlx.xsd should just be removed entirely, and the diagnostic message if the layer transform is not found should list all the currently registered layer transform names.Provide a way that the evaluation of dfdlx:layerTransform property, if the name does not correspond to a registered LayerTransformFactoryLayerCompiler, it could guess the name of the corresponding LayerTransformFactory object, (e.g, by uppercase first letter, and append "LayerTransformFactory" or "TransformFactory" if the name ends in "Layer", to the string value of the property) and try to dynamically load that class.dynamically load a class corresponding to it, using the service loader techniques already in use for UDFs and loadable validators. 

This mechanism would also eliminate the need for API code to explicitly call the LayerTransformFactorythe LayerCompilerRegistry.register(...) method. However, it is good practice to do that anyway, so as to get a Java linkage error (build/compile time of the software) rather than a dynamic runtime error. 

...

The 'daffodil-runtime1-layers' Module

The following existing (as of Daffodil 3.1.0) layering transforms should be removed from built-in layering transforms are found in the daffodil-runtime1 and put in a daffodil-layers module.

...

• fourByteSwap
• base64_MIME
• gzip
• lineFolded_IMF
• lineFolded_iCalendar

These enums for LayerTransformEnum should be removed as they are not implemented:

• quotedPrintable
• compress
• base64
• base64url

...

In daffodil-test there are tests for the above built-ins, but also examples of separately defined layer transforms.

• aisPayloadArmoring 
• IPv4Checksum - stripped down version of IPv4 checksums
• CheckDigits - check digit calculation in the style of the "OTH-Gold" data format. 

Documentation

A web page on the Daffodil user site should contain links to doc about each layer transform that is part of the daffodil-layers jar. 

Scaladoc(?) of each Layer Transform and of the Layering API. 

• These links should be to the scaladoc pages for each of these transforms.
• Each  transform's scaladoc should be where the transforms are thoroughly documented. 
• ??? There are pros/cons here.... needs discussion 
  • scaladoc is created in src/main/scala and it would end up almost repeating the example code which is in the src/test/resources and src/test/scala directories

Test Automation Plan

Testing during Daffodil 'sbt test' Build Cycle

Tests in daffodil-test will illustrate how dynamic loading of a layer from a jar works, but the jar in that case isn't a published jar.

This uses stripped down versions of the IPv4 header checksum, and OTH-Gold-style check-digits. 

• Improvement: Be sure to call these FakeIPv4 or similar renaming to avoid confusion with the real ethernet-ip schemas. 

...

Testing of User-like Situations - Outside the Daffodil Repository Tree

...