THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Motivation
ConsoleConsumer (kafka-console-consumer) is a very important debugging tool in Kafka. However, currently it cannot print offset, partition and headers of a Kafka record. So, during debugging session, if we need to know those information, we have to use kafka-dump-log, which require file system access to the Kafka broker hosts, or use custom application, like kafkacat. This KIP is proposing to rectify this issue.
Currently, without any extra property, ConsoleConsumer prints the value of the record.
And the following properties is one of the main debugging tools used by Kafka users. Especially for new users and new setups. However, not all possible message (ConsumerRecord) attributes are accessible using it.
Currently we have following parameters which control the ConsoleConsumer output format:
- print.timestamp
- print.key
- print.value
- key.separator
- line.separator
- key.deserializer
- value.deserializer
So As you see, there is no way to print message offset, partition and headers. Also, there is no easy way to differentiate null value or blank string.
...
New properties to DefaultMessageFormatter which can be specified in kafka-console-consumer command line arguments with --property argument.
| Property | Valid Values | Default Value | Description |
|---|---|---|---|
| print.offset | "true" or "false" | "false" | print message offset |
| print.partition | "true" or "false" | "false" | print message partition |
| print.headers | "true" or "false" | "false" | print headers |
| header.separator | string | "," | separator printed between each header's key:value pair |
| headers.deserializer | class name | "StringDeserializer" | header value deserializer |
| null.literal | string | "" (i.e. empty string) | literal to print if the value is NULL (size -1) |
Proposed Changes
Add new properties to DefaultMessageFormatter as per the above:
...