You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Status

Current state: Voting in progress

Discussion thread: here

JIRA: KAFKA-6733

PR: #4807

Motivation

ConsoleConsumer is one of the main debugging tools used by Kafka users. Especially for new users and new setups. However, not all possible message (ConsumerRecord) attributes are accessible using it.
Currently we have following parameters which control the ConsoleConsumer output format:

  • print.timestamp
  • print.key
  • print.value
  • key.separator
  • line.separator
  • key.deserializer
  • value.deserializer

So there is no way to print message offset, partition and headers. Also, there is no easy way to differentiate null value or blank string.


Public Interfaces

New properties to DefaultMessageFormatter which can be specified in kafka-console-consumer command line arguments with --property argument.

PropertyValid ValuesDefault ValueDescription
print.offset"true" or "false""false"print message offset
print.partition"true" or "false""false"print message partition
print.headers"true" or "false""false"print headers
header.separatorstring","separator printed between each header's key:value pair
headers.deserializerclass name"StringDeserializer"header value deserializer
null.literalstring"" (i.e. empty string)literal to print if the value is NULL (size -1)


Proposed Changes

Add new properties to DefaultMessageFormatter as per the above:

  • print.offset
  • print.partition
  • print.headers
  • header.separator
  • headers.deserializer
  • null.literal

Compatibility, Deprecation, and Migration Plan

The changes are backward compatible. If a user won’t use any new parameters, then the output of console consumer will look the same as before.

Rejected Alternatives

  • Use kafka-dump-log command to get these information. Rejected because this requires file system access to the Kafka broker's log.dirs, which is not secure.
  • Use other console consumer application to access these information. Rejected because Kafka brokers may run in airtight environment or as containers, preventing us from installing other applications than the one coming with Kafka brokers.
  • No labels