(Work in progress)

See also basic layering documentation: Proposal: Data Layering for Base64, Line-Folding, Compression, Etc

Extension of it for Checksums/CRC/Parity: Proposal: Checksums, CRC, Parity - Layering Enhancements

A Basic Mechanism Exists

As part of the work for Unable to render Jira issues macro, execution error. , there are new test layers in the daffodil-test module.

These include an IPv4 checksum layer example/test,  a checkDigits example/test, and AIS Payload Armoring example/test. 

These illustrate a way of doing "field extensibility" of layer transforms now, without any particular dynamic layer loading feature.

You pass the name of your layer transform using property

dfdlx:layerTransform="someLayer"

That name must be the name associated with a class derived from LayerCompiler.

LayerCompilers are dynamically loaded service providers. See the META-INF/services/org.apache.daffodil.layers.LayerCompiler files in daffodil-runtime1-layers/src/main/resources, and daffodil-test/src/test/resources

The 'daffodil-runtime1-layers' Module

The following existing built-in layering transforms are found in the daffodil-runtime1-layers module.

  • fourByteSwap
  • base64_MIME
  • gzip
  • lineFolded_IMF
  • lineFolded_iCalendar

In daffodil-test there are tests for the above built-ins, but also examples of separately defined layer transforms.

  • aisPayloadArmoring 
  • IPv4Checksum - stripped down version of IPv4 checksums
  • CheckDigits - check digit calculation in the style of the "OTH-Gold" data format. 

Documentation

There are two kinds:

  • user documentation for a DFDL schema author on how to use one of the built-in layers.
  • developer documentation for how to define a new layer transform for use in a data format DFDL schema. 

User documentation should be via a web page on the Daffodil user site. 

Developer documentation should be within the scaladoc of the Layering API classes. 

  • TBD: How to capture and exhibit this scaladoc for user-developers. 


Test Automation Plan

Testing of User-like Situations - Outside the Daffodil Repository Tree

DFDL Schemas Regression Testing Rig 

There is a general schema regression testing rig that enables developers to easily maintain a large tree of DFDL schemas for regression testing against daffodil. 

This tool/repo should be published. 

TODO: Create Apache Daffodil project repository for this.

This uses git submodules so that it does not copy schemas.

This should then test the "real" schemas mentioned below.

Updating "Real" DFDL Schema Examples that use Layers

The ethernet-ip DFDL schema (on github) can be made into a full exercise of  IPv4 checksums.

  • It could further be extended to do ICMP, TCP, UDP checksums also. 
  • The PCAP schema which uses this ethernet-ip schema would inherit this also. 

The OpenDFDL (on github - note: should move to DFDLSchemas) example checkDigits schema can be made into a complete example.

The DFDLSchemas (on github) GPS-SPS example computes parity bits. This could also be made into a complete test example - though this is quite large.



  • No labels