Status

Current state: Under Discussion

Discussion thread: here

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently tools in Kafka have three flavor of command line arguments:

command line arguments using dot, e.g. --consumer.config
command line arguments using dash --broker-list
command line arguments using position

#2 and #3 conforms to Unix standards while #1 does not.

Also, some tools always require config files for producer and consumer even when most people might just use default settings (ex. MirrorMaker).

And finally, some tools use a unique command line argument name for a command argument (ex. VerifyConsumerRebalance uses –zk to specify ZooKeeper URL while other tools use --zookeeper).

Proposed Changes

Change all tools using argument style #2 (arguments containing dots) to argument style #3 (following Unix standard, arguments containing dashes) (details listed below).
(Suggested by Gwen) Since mirror maker is more like a service instead of a tool. It makes sense to pass in a mirror maker config file.
Change unique/one-off command line argument names to match standard argument name (details listed below).

Command	Old Argument Name	New Argument Name	Reason For Change
kafka-console-consumer.sh	consumer.config	consumer-config	Not Unix Standard
kafka-consumer-offset-checker.sh	socket.timeout.ms	socket-timeout-ms	Not Unix Standard
kafka-consumer-offset-checker.sh	retry.backoff.ms	retry-backoff-ms	Not Unix Standard
ExportZkOffsets.scala	zkconnect	zookeeper	One-off Name
ImportZkOffsets.scala	zkconnect	zookeeper	One-off Name
KafkaMigrationTool.java	consumer.config	consumer-config	Not Unix Standard
KafkaMigrationTool.java	producer.config	producer-config	Not Unix Standard
KafkaMigrationTool.java	num.producers	num-producers	Not Unix Standard
KafkaMigrationTool.java	num.streams	num-streams	Not Unix Standard
KafkaMigrationTool.java	queue.size	queue-size	Not Unix Standard
kafka-mirror-maker.sh	consumer.config	consumer-config	Not Unix Standard
kafka-mirror-maker.sh	producer.config	producer-config	Not Unix Standard
kafka-mirror-maker.sh	num.streams	num-streams	Not Unix Standard
kafka-producer-perf-test.sh	request-num-acks	request-required-acks	One-off Name
kafka-replay-log-producer.sh	inputtopic	input-topic	One-off Name
kafka-replay-log-producer.sh	outputtopic	output-topic	One-off Name
kafka-replica-verification.sh	topic-white-list	whitelist	One-off Name
kafka-replica-verification.sh	report-interval-ms	reporting-interval	One-off Name
kafka-simple-consumer-shell.sh	fetchsize	fetch-size	One-off Name
TestLogCleaning.scala	zk	zookeeper	One-off Name
VerifyConsumerRebalance.scala	zookeeper.connect	zookeeper	One-off Name
TestLinearWriteSpeed.scala	compression	compression-codec	One-off Name
TestOffsetManager.scala	thread-count	threads	One-off Name
TestOffsetManager.scala	reporting-interval-ms	reporting-interval	One-off Name

Note that the following additional command line argument standardizations will be tracked via KAFA-2111 but are not part of this KIP since they are less invasive:

Add 'help' command line argument to any top-level tool scripts that were missing it.
Properly format argument descriptions (into sentences) and add any missing "REQUIRED" notes.

Compatibility, Deprecation, and Migration Plan

What impact (if any) will there be on existing users? - Previously used command line arguments will no longer function so end-users will need to adapt their calls when upgrading Kafka.
If we are changing behavior how will we phase out the older behavior? - There will be no deprecation of old command line arguments, only a change-over.
If we need special migration tools, describe them here. - No migration tools needed but the Kafka Documentation & Wiki pages will need to be updated to educate end-users on these changes.
When will we remove the existing behavior? Existing behavior will change as soon as patch is committed and included in release build.

Rejected Alternatives

None