THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Status
Current state: [One of "Under Discussion", "Accepted", "Rejected"]
Discussion thread: here TBD
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Currently all the tools in Kafka we have three flavor of command line arguments:
- command line arguments using dot, e.g. --consumer.config
- command line arguments using dash --broker-list
- command line arguments using position
2) #2 and 3) #3 conforms to Unix statndard standards while 1) #1 does not. We want to standardize them.
It would also be good if we can check whether it makes sense to change the interface so that the tools would be easier to use. e.g. mirror maker Also, some tools always require config files for producer and consumer even when most people might just use default settings (ex. MirrorMaker).
Finally, some tools use a unique command line argument name for a command argument (ex. VerifyConsumerRebalance uses –zk to specify ZooKeeper URL while other tools use --zookeeper).
Public Interfaces
The following list summarizes the flavor of each tool we have now in Kafka. The argument style are marked in parentheses using the number in previous section.
...
VerifyConsumerRebalacne(1)
Proposed Changes
- Change ConsoleConsumer, VerifyConsumerRebalance tools to use style 2).
- (Suggested by Gwen) Since mirror maker is more like a service instead of a tool. It makes sense to pass in a mirror maker config file.
...
- Standardize command line argument naming by making the following argument name changes:
| Command | Old Argument Name | New Argument Name |
|---|---|---|
| kafka-console-consumer.sh | consumer.config | consumer-config |
| kafka-consumer-offset-checker.sh | socket.timeout.ms | socket-timeout-ms |
| kafka-consumer-offset-checker.sh | retry.backoff.ms | retry-backoff-ms |
| ExportZkOffsets.scala | zkconnect | zookeeper |
| ImportZkOffsets.scala | zkconnect | zookeeper |
| KafkaMigrationTool.java | consumer.config | consumer-config |
| KafkaMigrationTool.java | producer.config | producer-config |
| KafkaMigrationTool.java | num.producers | num-producers |
| KafkaMigrationTool.java | num.streams | num-streams |
| KafkaMigrationTool.java | queue.size | queue-size |
| kafka-mirror-maker.sh | consumer.config | consumer-config |
| kafka-mirror-maker.sh | producer.config | producer-config |
| kafka-mirror-maker.sh | num.streams | num-streams |
| kafka-producer-perf-test.sh | request-num-acks | request-required-acks |
| kafka-replay-log-producer.sh | inputtopic | input-topic |
| kafka-replay-log-producer.sh | outputtopic | output-topic |
| kafka-replica-verification.sh | topic-white-list | whitelist |
| kafka-replica-verification.sh | report-interval-ms | reporting-interval |
| kafka-simple-consumer-shell.sh | fetchsize | fetch-size |
| TestLogCleaning.scala | zk | zookeeper |
| VerifyConsumerRebalance.scala | zookeeper.connect | zookeeper |
| TestLinearWriteSpeed.scala | compression | compression-codec |
| TestOffsetManager.scala | thread-count | threads |
| TestOffsetManager.scala | reporting-interval-ms | reporting-interval |
Note that the following additional command line argument standardizations will be tracked via KAFA-1293 but are not part of this KIP since they are less invasive:
- Add 'help' command line argument to any top-level tool scripts that were missing it.
- Properly format argument descriptions (into sentences) and add any missing "REQUIRED" notes.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users? - Previously used command line arguments will no longer function.
- If we are changing behavior how will we phase out the older behavior? - End-users will need to adapt.
- If we need special migration tools, describe them here. - None but note that Kafka Documentation & Wiki pages will need to be updated.
- When will we remove the existing behavior? Users building from
Rejected Alternatives
None