Status

Current state: Under Discussion

Discussion thread: Here

JIRA: KAFKA-7711 

PR: KAFKA-7569

Motivation

Currently the call to Producer.flush() can be hang there for indeterminate time.

It would be great to add a bounded flush() API and timeout if producer is unable to flush all the batch records in a limited time. In this way the caller of flush() has a chance to decide what to do next instead of just wait forever.

Public Interfaces

A new API will be added to Kafka producer

KafkaProducer
@Override
public void flush(Duration timeout)

This API returns normally if flush succeed in time, i.e all previously sent records have completed ; otherwise if times out before finish sending all records, an TimeoutException will be thrown.

Proposed Changes

Before this proposal, there is only one flush() API, which the caller will be blocked util all the pending batches(at the moment of calling flush()) succeed or exhaust all the retries.


After this proposal, the caller can specify an upper bound of the waiting time, and get notified wether flush succeed in this bounded time.

When timeout happens, this blocking call returns, but the sending/retry of incomplete batches does not get affected. 

Compatibility, Deprecation, and Migration Plan

There is not compatibility concern or migration plan needed for this change.

Rejected Alternatives

None.

  • No labels