Table of Contents |
---|
Status
Current state: Under Discussion
Status | ||||
---|---|---|---|---|
|
Discussion thread: here
Vote Discussion thread: here
JIRA:
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Pull Request: here
Motivation
Tools kafka-console-producer.sh
and kafka-console-consumer.sh
have the multiple option --property
to setup the MessageReader
for Producer (defined by --line-reader
) or the MessageFormatter
for Consumer (defined by --formatter
).
...
The idea is to offer the same possibility to use a config file option to setup the MessageReader
/MessageFormatter
in addition to the existing option --property
.
It will be useful in different use cases:
...
As for --producer-property
/--consumer-property
with --consumer.config
/--producer.config
, any value from option --property
would override value from config file.
Simple example
Before (multiple
--property
options):Code Block language bash kafka-console-producer.sh [...] --property print.timestamp=true \ --property print.key=true \ --property print.offset=true \ --property print.partition=true \ --property print.headers=true \ --property print.value=true \ --property key.separator=':' \ --property line.separator=';' \ --property headers.separator='|'
After (all properties in one file):
Code Block language actionscript3 title reader-config.properties print.timestamp=true print.key=true print.offset=true print.partition=true print.headers=true print.value=true key.separator=: line.separator=; headers.separator=|
Code Block language bash kafka-console-producer.sh [...] --reader-config reader-config.properties
Mixed example
When a property is in the config file and as option, option wins.
Code Block | ||||
---|---|---|---|---|
| ||||
print.timestamp=true print.key=true print.offset=true print.partition=true print.headers=true print.value=true key.separator=: line.separator=; headers.separator=| |
Code Block | ||
---|---|---|
| ||
kafka-console-producer.sh [...] --reader-config reader-config.properties --property key.separator='/' |
Then, the MessageReader
will get key.separator=/
, not key.separator=:
Proposed Changes
Implement the new options as described in Public Interfaces section:
ConsoleProducer
: new option "reader-config
" with description "Config properties file for the message reader. Note that --property takes precedence over this config.
"ConsoleConsumer
: new option "formatter-config
" with description "Config properties file to initialize the message formatter. Note that --property takes precedence over this config.
"
Already implemented in pull request #12175.
...