Page History

...

There is one new method exposed on the partitioner interface.

   /**
   * Executes right before a new batch will be created. For example, if a sticky partitioner is used,
   * this method can change the chosen sticky partition for the new batch. 
   * @param topic The topic name
   * @param cluster The current cluster metadata
   * @param prevPartition The partition of the batch that was just completed
   */
  default public void batchCompletedonNewBatch(String topic, Cluster cluster, int prevPartition) {
  }

The method batchCompleted onNewBatch will execute code right before a new batch is created. The sticky partitioner will define this method to update the sticky partition. This includes changing the sticky partition even when there will be a new batch on a keyed value. Test results show that this change will not significantly affect latency in the keyed value case.

...

These changes will slightly modify the code path for records that have keys as well, but the changes will not affect latency significantly.

A new partitioner called StickyRoundRobinPartitioner UniformStickyPartitioner will be created to allow sticky partitioning on all records, even those that have non-null keys. This will mirror how the RoundRobinPartitioner uses the round robin partitioning strategy for all records, including those with keys.

...

...