THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
Status
Current state: Voting in progress Accepted
Discussion thread: here
Voting thread: here
JIRA: KAFKA-6733
Motivation
ConsoleConsumer (kafka-console-consumer) is a very important debugging tool in Kafka. However, currently it cannot print offset, partition and headers of a Kafka record. So, during debugging session, if we need to know those information, we have to use kafka-dump-log, which require file system access to the Kafka broker hosts, or use custom application, like kafkacat. This KIP is proposing to rectify this issue.
...
- print.timestamp
- print.key
- print.value
- print.partition
- key.separator
- line.separator
- key.deserializer
- value.deserializer
...
| Property | Valid Values | Default Value | Description |
|---|---|---|---|
| print.offset | "true" or "false" | "false" | print message offset |
| print.headers | "true" or "false" | "false" | print headers, will print NO_HEADERS if the record has no headers |
| header.separator | string | "," | separator printed between each header's key:value pair |
| headers.deserializer | class name | "StringDeserializer" | header value deserializer |
| null.literal | string | "null" | literal to print if the value is NULL (size -1) |
...
| Code Block |
|---|
$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning --property print.partition=true --property print.key=true --property print.timestamp=true --property print.offset=true --property print.headers=true --property key.separator='|' CreateTime:1592475472398|null|asdf|Partition:0|Offset:3|h1=:v1,h2=v2 :v2|key1|value1 CreateTime:1592475472456|null|qwer|Partition:0|Offset:4|NO_HEADERHEADERS|key2|value2 |
Proposed Changes
Add new properties to DefaultMessageFormatter as per the above:
...
Compatibility, Deprecation, and Migration Plan
KIP-431 introduces incompatibility when "print.partition=true" (this property exists in the code before KIP-431 but not documented). Before KIP-431, "kafka-console-consumer" prints the partition as a number after the value for example: "key1|value1|0". After this KIP, "kafka-console-consumer" prints the partition number prefixed with "Partition:" before the key (if printed) and value, for example: "Partition:0|key1|value1" . Because this property was not documented, no migration plan is implemented in KIP-431.
The other changes are backward compatible . If a user won’t because they do not exist before. Apart from "print.partition=true", if a user does not use any new parameters, then the output of console consumer will look the same as before.
...