...

We propose adding a new timeout delivery.timeout.ms. The window of enforcement includes batching in the accumulator, retries, and the inflight segments of the batch. With this changeconfig, the user has a guaranteed upper bound on when a record will either get sent, fail or expire : max.block.ms + delivery.timeout.msfrom the point when send returns. In other words we no longer overload request.timeout.ms to act as a weak proxy for accumulator timeout and instead introduce an explicit timeout that users can rely on without exposing any internals of the producer such as the accumulator. 

...

This config enables applications to delegate error handling to Kafka to the maximum possible extent (by setting retries=MAX_INT and delivery.timeout.ms=MAX_LONG). And it enables MirrorMaker to bound the effect of unavailable partitions by setting delivery.timeout.ms to be sufficiently low, presumably some function of the expected throughput in the steady state. Specifically, setting delivery.timeout.ms to a minimum of request.timeout.ms + retry.backoff.ms + linger.ms, would allow at least one attempt to send the message when the producer isn't backed up. message.max.delivery.wait.ms – new config, controls how long to try to send messages before erroring them out. 

The "timer" for each batch starts "ticking" at the creation of the batch. Batches expire in order when max.in.flight.request.per.connection==1. An inflight batch expire when delivery.timeout.ms has passed since the batch creation irrespective of whether the batch is in flight or not.   

Public Interfaces

•  This only adds a new producer configuration: delivery.timeout.ms.

...

Validation

Test Plan

• Additional unit tests in o.a.k.c.p.i.RecordAccumulatorTest. 
• TBD

Rejected Alternatives

Anchor
rejected rejected

...