Page History

...

We will expose the following timeout in KafkaProducer:

1. max.enqueue.block.ms - the maximum time producer.send() and partitionsFor() will block, including.
   1. For send() it includes:
      1. metadata fetch time
      2. buffer full block time
      3. serialization time (customized serializer)
      4. partitioning time (customized partitioner)
   2. For partitionsFor() it includes:
      1. metadata fetch time
2. request.timeout.ms - the maximum time to wait for the response of a message AFTER the batch is ready,  including:
   
   1. actual network RTT
   2. server replication time

...

1. Request timeout.ms  only starts counting down after a batch is ready.
2. Request timeout.ms does not include retries - each try has a full request.timeout.ms. The reason we did not include retry here is that if we have the request timeout include retries, then we should actually keep retry until timeout. That indicates we should remove retry config but this causes problem for users who don't want duplicates. Also it is a little bit strange to timeout before user specified retries are exhausted.
3. Request timeout will also be used when : the batches in the accumulator that are ready but not drained due to metadata missing - we are reusing request timeout to timeout the batches in accumulator.When people call partitionsFor() - we are reusing request timeout here in a sense of "waiting for metadata response of a topic". It is a little bit strange but will save us another timeout configuration.

The benefits of this approach are:

...

public static final String MAX_ENQUEUEBLOCK_TIMEOUT_MS_CONFIG = "max.enqueue.timeoutblock.ms"
private static final String MAX_ENQUEUE_TIMEOUTBLOCK_MS_DOC = "The configuration controls how long {@link KafkaProducer#send()} will block." +  
                                                      "The send method can be blocked because of buffer full, metadata not available, " + 
                                                      "customized partitioner/serialzier."

...