Status

Current state: Under Discussion Accepted

Discussion thread: https://lists.apache.org/thread/vdp8scrrzdq7ofvl0mm84dhphq8kmzgc

...

In production, producers commonly send keyed messages records to leverage semantic partitioning for ordering guarantees, stream joins, and cross-cluster replication. However, kafka-producer-perf-test always produces records with null keys, making benchmark results systematically optimistic and unable to reflect the performance characteristics of real keyed workloads. Additionally, since log compaction requires non-null keys, the tool currently cannot benchmark compacted topics at all. Adding key support removes this limitation.

...

--key-distribution <none|range|random> (optional, default: none)

Controls how message record keys are assigned:

• none — null key (current behavior, default)
• range — keys cycle through integers 0, 1, ..., KEY-RANGE-1 in round-robin order
• random — each record gets a randomly selected integer from [0, KEY-RANGE)

--

...

record-key-range <KEY-RANGE> (optional, required when --key-distribution is range or random)

Defines the size of the key space. Must be a positive integer.

...

...

• Null keys — existing behavior (default)
  

  Code Block
  bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092

• Round-robin across 100 distinct keys
  

  Code Block
  bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092 \ --key-distribution range --messagerecord-key-range 100

• Random keys from a space of 10,000
  

  Code Block
  bin/kafka-producer-perf-test.sh \ --topic my-topic --num-records 1000000 --record-size 1024 \ --throughput -1 --bootstrap-server localhost:9092 \ --key-distribution random --messagerecord-key-range 10000

Compatibility, Deprecation, and Migration Plan

...

Engineers who genuinely need globally unique keys can use --key-distribution random --messagerecord-key-range <large-number> (e.g., 2^31−1) to approximate the same effect without the overhead.