Current state: Under Discussion
Discussion thread: here
JIRA: KAFKA-12392
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
The ConsoleProducer has two options, --batch-size and --max-partition-memory-bytes that both configure the underlying batch.size producer config. One of them should be removed.
A previous discarded KIP-717 suggested removing --batch-size . However, this KIP argues for the opposite. Retaining --batch-size is more straightforward and aligns better with the terminology used throughout Kafka's community.
This change affects the command-line interface of
kafka-console-producer.shThe option --max-memory-bytes will be marked as deprecated and then removed in next major release.Fix the description for --batch-size and add deprecated message for option --max-partition-memory-bytes
./kafka-console-producer.sh
This tool helps to read data from standard input and publish it to Kafka.
Option Description
------ -----------
--batch-size <Integer: size> Number of messages to send in a single
batch if they are not being sent
synchronously. please note that this
option will be replaced if max-
partition-memory-bytes is also set
(default: 16384)
...
--max-partition-memory-bytes <Integer: The buffer size allocated for a
memory in bytes per partition> partition. When records are received
which are smaller than this size the
producer will attempt to
optimistically group them together
until this size is reached. This is
the option to control `batch.size`
in producer configs. (default: 16384) |
./kafka-console-producer.sh
This tool helps to read data from standard input and publish it to Kafka.
Option Description
------ -----------
--batch-size <Integer: size> The buffer size in bytes allocated for a partition.
When records are received
which are smaller than this size the
producer will attempt to
optimistically group them together
until this size is reached. This is
the option to control `batch.size`
in producer configs. (default: 16384)
...
--max-partition-memory-bytes <Integer: (Deprecated) Please use --batch-size instead.
memory in bytes per partition> The buffer size allocated for a partition.
When records are received
which are smaller than this size the
producer will attempt to
optimistically group them together
until this size is reached. This is
the option to control `batch.size`
in producer configs. (default: 16384) |
--max-partition-memory-bytes with --batch-size in their scripts.N/A
--batch-sizeUse --max-memory-bytes to indicate batch.size in ProducerConfig is less intuitive.
--max-memory-bytes. Instead, use --command-config or --command-property to pass producer propertiesRejected because keeping the dedicated --batch-size flag is more convenient for users who frequently test the interaction between batch.size and linger.ms.