Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This proposal adds key distribution support to kafka-producer-perf-test, allowing engineers to benchmark keyed workloads with configurable key ranges and distribution strategies.

The three distribution modes cover the most common real-world keyed workload patterns:

  • none — establishes a null-key baseline for topics where ordering and co-partitioning are not required, such as log aggregation pipelines.
  • range — models workloads where a bounded, predictable set of keys cycles repeatedly, such as Kafka Streams joins or MirrorMaker 2 replication, where the same key must consistently land on the same partition to preserve ordering guarantees.
  • random — models workloads with a bounded but unpredictably distributed key space. A small key range (e.g., 10) relative to the partition count simulates hot-partition scenarios; a large range (e.g., 1,000,000) approximates unique-key workloads such as IoT device data without the overhead of UUID generation.

Public Interfaces

This proposal adds two new command-line arguments to kafka-producer-perf-test:

...

Defines the size of the key space. Must be a positive integer.

--random-seed <seed> (optional, default 0)

Controls the seed for the pseudo-random number generator used by --key-distribution random and random payload generation. The default value of 0 ensures deterministic, reproducible benchmark runs. Set to a different value when non-repeating sequences are required.

Proposed Changes

New Enum: KeyDistribution

...

  • Null keys — existing behavior (default)
    Code Block
    bin/kafka-producer-perf-test.sh \
      --topic my-topic --num-records 1000000 --record-size 1024 \
      --throughput -1 --bootstrap-server localhost:9092
  • Round-robin across 100 distinct keys
    Code Block
    bin/kafka-producer-perf-test.sh \
      --topic my-topic --num-records 1000000 --record-size 1024 \
      --throughput -1 --bootstrap-server localhost:9092 \
      --key-distribution range --message-key-range 100 --random-seed 10
  • Random keys from a space of 10,000
    Code Block
    bin/kafka-producer-perf-test.sh \
      --topic my-topic --num-records 1000000 --record-size 1024 \
      --throughput -1 --bootstrap-server localhost:9092 \
      --key-distribution random --message-key-range 10000 --random-seed 10

Compatibility, Deprecation, and Migration Plan

...