Versions Compared

Old Version 5

changes.mady.by.user Mike Beckerle

Saved on

compared with

New Version Current

changes.mady.by.user Mike Beckerle

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: update for vcard

Parts of this page have been incorporate4d into Open Grid Forum Experience Document: Interoperability Demonstration of IBM DFDL and Apache Daffodil (Incubating) Implementations. 

However, there are additional testing details here not included in that report.

This page is draft materials for an eventual OGF Experience Document describing interoperability testing with IBM DFDL.

...

Public DFDL Schemas

These schemas are published, and are interoperable:

  • CSV
  • NACHA
  • ISO8583
  • QuasiXML
  • GeoNames

The following schemas are published, are intended to be portable/interoperable, but do not yet work across implementations due to DAFFODIL features/bugs

  • IBM4690TLOG
  • EDIFACT

The following schemas are published, but need minor adjustments for portability, including possibly documentation edits on the DFDLSchemas web site materials.

The following schemas are published, but make use of optional DFDL language features available in only Daffodil, and so are not expected to interoperate:

...

The following schemas are listed on DFDLSchemas, but are available only under license from IBM, and are not candidates for interoperability testing:

  • HIPAA-5010
  • HL7-v2.7 (note: an earlier no-longer-published revision may be available for testing)
  • SWIFT-MT
  • HL7 - maintained versionsSWIFT-MT

Non-Public DFDL Schemas

These schemas are available only to those with access to DI2E.net or Forge.mil; however, as that is a substantial community of users, we also document interoperability for these schemas here:

Unknown/To-be-Evaluated

These schemas should be able to be portable, but have not yet been evaluated.

  • agnosc_remedy
  • army_drrs
  • common_event_format
  • uscg_ucop
  • USMTF

Portable (works on Daffodil and IBM DFDL)

  • TBD

Expected Non-Portable - works only on Daffodil due to use of Daffodil-specific features

  • VMF

  • GMTIF

...

Portable (works on Daffodil and IBM DFDL)

  • cef - common-event-format - works for parse and unparse.
  • army_drrs - works for parse and unparse.
  • uscg_ucop  - Works for parse and unparse.

Expected Non-Portable

These schemas use optional DFDL features available only in Daffodil.

  • agnosc_remedy - schema makes use of asserts that invoke dfdl:checkConstraints - not supported by IBM DFDL (as of 2019-05-28)
    • Was tested for parse with these asserts removed. - Parse gives error separator not found. However, it is not expected that this would work.
  • VMF - uses dfdl:inputValueCalc and dfdl:outputValueCalc extensively, as well as dfdl:bitOrder="leastSignificantBitFirst"

  • GMTIF - uses dfdl:inputValueCalc extensively

  • USMTF - A few small tests pass, but this schema makes extensive use of backtracking via assertions which invoke dfdl:checkConstraints - not supported by IBM DFDL (as of 2019-05-28)

Eventually Public DFDL Schemas

These schemas are restricted access pending permission to make public. Their status w.r.t., portability is

...

as noted.

  • iCalendar - uses dfdl:inputValueCalc (daffodil only as of 2019-05-28) - possibly this could be removed.
  • IMF - unknown
  • iCalendar
  • IMF
  • IPFIX

Test Suites

Test Suite 'daffodil-test-ibm1'

This test suite is part of the Daffodil source tree, but can be run against IBM DFDL using a cross tester.

Daffodil tests that fail on IBM DFDL, but NOT due to interoperability concerns:

These Daffodil tests fail on IBM_DFDL, but should not. They fail because of DAFFODIL-2026 - a limitation in Daffodil's TDML Tester

  • test_simple_type_properties_text_calendar_13_03
  • test_simple_type_properties_text_calendar_13_04
  • test_simple_type_properties_bin_calendar_13_01
  • test_simple_type_properties_bin_calendar_13_02
  • test_simple_type_properties_text_calendar_13_01
  • test_alignment_bytes_12_05

Incompatibilities

These tests fail for lack of support of identical separator and terminator in IBM DFDL. All these are the same issue. This issue is considered minor, and is not a barrier to interoperability in practical schemas.

  • test_length_delimited_12_01
  • test_length_delimited_12_04
  • test_multiple_delimiters
  • test_length_delimited_12_02
  • test_length_delimited_12_03
  • test_nested_separator_delimited_baseline_ibm

Needs Further Investigation

Currently (2018-12-10) These tests fail on IBM DFDL with a NPE during schema compilation. No trace output is produced  - Need study to determine cause. The origin of these tests was the very earliest part of the Daffodil project at Univ. of Illinois NCSA.

  • test_AF000, test_AF001, test_AF002
  • test_AW000, test_AW001
  • test_AX000
  • test_AY000
  • test_AZ000

All tests in this suite run portably on Daffodil and IBM DFDL. (2019-05-17 development branch).

A few tests are conditionalized and run on only one or the other due to specific features that Daffodil has, or that IBM has.

Daffodil Test Suite 'daffodil-test'

This is a larger test suite.

...

1104 fail as of 2018-12-10

Common Test Failures To Be Resolved

The vast bulk of the failures can be explained by a small number of structural problems with the test suite or test infrastructure:

  • daf:error - function obviously not supported by IBM DFDL.
  • Many daffodil tests put constructs on global elements that are not allowed on global elements by DFDL.
  • Many tests have namespace errors - root elements of infosets are not in the proper namespace.  (commonly this is the ex or http://example.com namespace)
  • Missing properties (textBooleanTrueRep/FalseRep being undefined is one)
  • Mixing of invalid schema errors in the same test suites with other tests.
    • Different defineSchema inside a TDML file are not being isolated - compile errors in one stop all tests in that TDML file from compiling.
    • This is a TDML Runner issue and just how tests are authored.
  • Incorrect property syntax: extraEscapedCharacter="%#WSP*; ..." for example
  • Missing properties: textBidi, textPadKind, textTrimKind, escapeSchemeRef, documentFinalTerminatorCanBeMissing, and many others are missing from  test environments.

Incompatibilities

  • test_timeFunctions01 - IBM DFDL does not support dfdl:calendarTimeZone="" (empty string, meaning 'no time zone')
    • This is problematic because the default format most tests use has this property defined this way.
    • TBD: Perhaps tests should choose from the normal daffodil default formats, and one designed for compatibility??
  • dfdl:calendarPattern contains more than 3 "S" symbols. (Should be a warning? - possible bug in cross-test rig)
  • separator and escapeCharacter cannot include the same value - this is tolerated by Daffodil. (Allows a delimiter to be escaped by doubling it.)

Incompatibilities Which May be Daffodil Bugs

  • When type is long, int, short or byte, DFDL property 'binaryNumberRep' must not be 'bcd' as this cannot have a negative value. (testBCDCharset01)
    • IBM DFDL insists on signed types not being BCD since it has no way to represent negative values.
  • Daffodil tolerates not having emptyValueDelimiterPolicy set, if it can show there is no concept of "empty" for the definition. This is not portable because the rules in DFDL are simpler than this. The property simply is required if terminator is defined. (or initiator), regardless of whether there is any notion of empty for a type.
  • truncateSpecifiedLengthString - missing
  • trailingSkip - missing
  • alignment - missing

...