Status

Current state: Approved

Discussion thread: here

Voting thread: here

Motivation

Currently it is not possible to add headers to records when using the kafka-console-producer. The kafka-console-producer is an important debugging tool: The possibility to add headers to a record will improve the debugging experience.

Public Interfaces

New properties to LineMessageReader

Property	Valid Values	Default Value	Description
parse.headers	"true" or "false"	"false"	parse header information
headers.delimiter	string	"\t"	delimits where header input for each line ends
headers.separator	string	","	separator between header key-value-pair
headers.key.separator	string	":"	separates the header's key from the header's value

Default parsing pattern

when parse.headers=true and parse.key=true:
"h1:v1,h2...\tkey\tvalue"

when parse.headers=true and parse.key=false:
"h1:v1,h2...\tvalue"

when parse.headers=false and parse.key=true:
"key\tvalue"

when parse.headers=false and parse.key=false :
"value"

$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:29092 --topic test --property parse.key=true --property parse.headers=true
>h0:v0,h1:v1 key value

Proposed Changes

Add new properties to ConsoleProducer.LineMessageReader as per the above:

parse.headers
headers.delimiter
headers.separator
headers.key.separator

Compatibility, Deprecation, and Migration Plan

What impact (if any) will there be on existing users?

The change is backward compatible. Current users of the kafka-console-producer can use it as they did so far.

If we are changing behavior how will we phase out the older behavior?

Not applicable.

Rejected Alternatives

Use other console producer like 'kafkacat'. This introduces additional dependencies whereas the proposed solution does not.