Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Table of Contents

Status

Current state: Under Discussion  Accepted

Discussion threadhere

Vote thread: here

...

For example, you could use the config:

Code Block
schemas.enable: =true
schema.content: '={"type": "struct", "fields": [{ "field": "id", "type": "string", "optional": false },{ "field": "name", "type": "string", "optional": false }]}'

and the Kafka message payload contains:

...

Users would be able use ConfigProviders to provide the contents of a file as a schema.

For Example, 

Code Block
# Config for Kafka Connect
config.providers=directory
config.providers.directory.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider

# Config for Kafka Convertor
schema.content:=${directory:/schema/:schema.json}

KIP-993: Allow restricting files accessed by File and Directory ConfigProviders will allow users to restrict what files to be used.


While using Connect REST API to create Connector with Schema Configuration, we will need include escape sequences in the schema.content field since REST API payload needs to be a JSON.

For Example

Code Block
{
  "name": "api-file-source-json",
  "config": {
    "value.converter.schemas.enable": true,
    "value.converter.schema.content": "{\"type\": \"struct\", \"fields\": [{ \"field\": \"id\", \"type\": \"string\", \"optional\": false },{ \"field\": \"name\", \"type\": \"string\", \"optional\": false }]}"
  }
}


Proposed Changes

Functional changes will be made to org.apache.kafka.connect.json.JsonConverter  configure

...