DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
This proposal adds key distribution support to kafka-producer-perf-test, allowing engineers to benchmark keyed workloads with configurable key ranges and distribution strategies.
The three distribution modes cover the most common real-world keyed workload patterns:
- none — establishes a null-key baseline for topics where ordering and co-partitioning are not required, such as log aggregation pipelines.
- range — models workloads where a bounded, predictable set of keys cycles repeatedly, such as Kafka Streams joins or MirrorMaker 2 replication, where the same key must consistently land on the same partition to preserve ordering guarantees.
- random — models workloads with a bounded but unpredictably distributed key space. A small key range (e.g., 10) relative to the partition count simulates hot-partition scenarios; a large range (e.g., 1,000,000) approximates unique-key workloads such as IoT device data without the overhead of UUID generation.
Public Interfaces
This proposal adds two new command-line arguments to kafka-producer-perf-test:
...
Defines the size of the key space. Must be a positive integer.
--random-seed <seed> (optional, default 0)
Controls the seed for the pseudo-random number generator used by --key-distribution random and random payload generation. The default value of 0 ensures deterministic, reproducible benchmark runs. Set to a different value when non-repeating sequences are required.
Proposed Changes
New Enum: KeyDistribution
...
- Null keys — existing behavior (default)
Code Block bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092
- Round-robin across 100 distinct keys
Code Block bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092 \ --key-distribution range --message-key-range 100 --random-seed 10 - Random keys from a space of 10,000
Code Block bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092 \ --key-distribution random --message-key-range 10000 --random-seed 10
Compatibility, Deprecation, and Migration Plan
...