You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 17 Next »

Status

Current state: Under Discussion

Discussion thread: here

Voting thread: here

JIRA: KAFKA-13351

PR: #11456

Motivation

Currently it is not possible to add headers to records when using the kafka-console-producer. The kafka-console-producer is an important debugging tool: The possibility to add headers to a record will improve the debugging experience.

Public Interfaces

New properties to LineMessageReader

PropertyValid ValuesDefault ValueDescription
parse.headers"true" or "false""false"parse header information
headers.delimiterstring

"\t"

delimits where header input for each line ends
headers.separator

string

","separator between header key-value-pair
headers.key.separatorstring":"separates the header's key from the header's value


Default parsing pattern

when parse.headers=true and parse.key=true:
"h1:v1,h2...\tkey\tvalue"

when parse.headers=true and parse.key=false:
"h1:v1,h2...\tvalue"

when parse.headers=false and parse.key=true:
"key\tvalue"

when parse.headers=false and parse.key=false :
"value"


$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:29092 --topic test --property parse.key=true --property parse.headers=true
>h0:v0,h1:v1 key value


Proposed Changes

Add new properties to ConsoleProducer.LineMessageReader as per the above:

  • parse.headers
  • headers.delimiter
  • headers.separator
  • headers.key.separator

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?

    The change is backward compatible. Current users of the kafka-console-producer can use it as they did so far.


  • If we are changing behavior how will we phase out the older behavior?

    Not applicable.

Rejected Alternatives

  • Use other console producer like 'kafkacat'. This introduces additional dependencies whereas the proposed solution does not.
  • No labels