Versions Compared

Old Version 35

changes.mady.by.user Jorge Otoya

Saved on

compared with

New Version 36

changes.mady.by.user Jorge Otoya

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Single Message Transforms (SMT),  introduced withKIP-66, have greatly improved Connector's usability by enabling transforming transformation of input/output data format formats and content contents without the need for additional streaming applications. 

ThoughHowever, these benefits have been limited by SMTs constraint limitation to only lookup for act on fields available on at the root of the data structure.

Here are some tickets/comments related to this limitation:

This KIP aim aims to include support for nested structures on the existing SMTs.

...

scenarioinputsmtoutput
1. Nested field.


Code Block
languagejs
{
  "k1": 123,
  "parent": {
    "child": {
      "k2": 1556204536000         }
  }
}



Code Block
languagejs
{
"transforms": "smt1",
"transforms.smt1.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value", "transforms.smt1.field.syntax.version": "v2",
"transforms.smt1.field": "parent.child.k2",
"transforms.smt1.format": "yyyy-MM-dd",
"transforms.smt1.target.type": "string"
}



Code Block
languagejs
{
  "k1": 123,
  "parent": {
    "child": {
      "k2": "2014-04-25"         }
  }
}


2. Nested field, when field names include dots


Code Block
languagejs
{
  "k1": 123,
  "parent.child": {
      "k2": 1556204536000         }
  }
}



Code Block
languagejs



{
"transforms": "smt1",
"transforms.smt1.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value", "transforms.smt1.field.syntax.version": "v2",
"transforms.smt1.field": "`parent.child`.k2",
"transforms.smt1.format": "yyyy-MM-dd",
"transforms.smt1.target.type": "string"
}



Code Block
languagejs
{
  "k1": 123,
  "parent.child": {      "k2": "2014-04-25"   }
}


...