You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Status

Current state: Under Discussion

Discussion thread: here

Voting thread: here

JIRA: KAFKA-13351

PR: #11456

Motivation

Currently it is not possible to add headers to records when using the kafka-console-producer.

Public Interfaces

New properties to LineMessageReader

PropertyValid ValuesDefault ValueDescription
parse.headers"true" or "false""false"parse header information
headers.delimiterstring

"\t"

delimits where header input for each line ends
headers.separator

string

","separator between header key-value-pair
headers.key.separatorstring":"separates the header's key from the header's value


Default parsing pattern

when parse.headers=true and parse.key=true:
"h1:v1,h2...\tkey\tvalue"

when parse.headers=false and parse.key=true:
"key\tvalue"

when parse.headers=true and parse.key=false:
"h1:v1,h2...\tvalue


$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:29092 --topic test --property parse.key=true --property parse.headers=true
>h0:v0,h1:v1 key value


Proposed Changes

Add new properties to ConsoleProducer.LineMessageReader as per the above:

  • parse.header
  • headers.delimiter
  • headers.separator
  • headers.key.separator

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?

    The change is backward compatible. Current users of the kafka-console-producer can use it as they did so far.


  • If we are changing behavior how will we phase out the older behavior?

    Not applicable.

Rejected Alternatives

  • Use other console producer like 'kafkacat'. This introduces additional dependencies whereas the proposed solution does not.
  • No labels