Page History

...

The sticky partitioner will be part of the default partitioner, so there will not be a public interface directly.

There are two is one new methods method exposed on the partitioner interface. There will also be one new class. 

    /**
 * Computes a partition and other relevant *data. RunsKnows beforewhether thea new batch iswill be created iffor willCallOnNewBatchthe returns truerecord.
      * 
 * @param topic The topic name
 * @param topickey The topic name
     key to partition on (or null if no key)
 * @param keyBytes The serialized key ofto thepartition record on( or null if no key)
 * @param value The value to partition on or null
 * @param valueBytes The serialized value ofto thepartition recordon or null
      * @param cluster The current cluster metadata
 * @param isNewBatch A boolean to specify whether a new batch */
    default void onNewBatchwill be created
 * @return ComputedPartition object to specify details about the record's partition
 */
default ComputedPartition computePartition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster, boolean isNewBatch) {
    return new ComputedPartition(partition(topic, key, keyBytes, value, valueBytes, returncluster));
}

    }/**
     * Stores both the partition and whether computePartition should be called when a new batch
    /**
 * is about to be created.
     */
    class ComputedPartition{
       * Determines whether to call onNewBatch based on the key, value, and whether there is an explicit partition
     *
     * @param keyBytes The serialized key of the record ( or null if no key)
     * @param valueBytes The serialized value of the record or null int part;
        boolean willCallOnNewBatch;
        
        ComputedPartition(int part) {
             this.part = part;
             this.willCallOnNewBatch = false;
        }
        
     *  @param explicitPartition A boolean to represent whether an explicit partition id has been provided
ComputedPartition(int part, boolean willCallOnNewBatch) {
             this.part = part;
             this.willCallOnNewBatch = willCallOnNewBatch;
        */}
      default  
   boolean willCallOnNewBatch(byte[] keyBytes, byte[] valueBytes, boolean explicitPartitionwillCallOnNewBatch() { return this.willCallOnNewBatch; }
        int get() { return falsethis.part; }
    }

When adding a record and a new record batch is created, the result of willCallOnNewBatch will determine whether onNewBatch is called. onNewBatch is called before the new batch is created, so the partition can be changed before the batch is made.

The sticky partitioner will use these methods to call onNewBatch for records with no keys and no explicit partitions, and switch the sticky partition on the new batch.

The method computePartition will replace partition in the KafkaProducer. It will return a computePartition object that contains both the partition and a boolean that determines whether we call the method again exactly once for a record if that record would have created a new batch.

The sticky partitioner will define this method so that when we have non-keyed, no explicit partition-id records, willCallOnNewBatch will be true and we will update the sticky partition when there is a new batch. All other cases will have the expected behavior.

The default of this method The default will result in no change to the current behavior for other user-defined partitioners. If a user wants to implement a sticky partitioner in their own partitioner class, these this method can be overridden.

...