Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

the new KafkaConsumer#commitSync will accept user-specified timeout.  

Proposed Changes

One change shall be made to KafkaConsumer#poll() whose calls in Kafka internals  (specifically fetchCommittedOffsets() ) will block indefinitely. This is the exact same problem as what other methods face (such as committed() and position()). To avoid blocking indefinitely, the new methods will take user-determined timeout to define the maximum amount of time for which a method blocks.

...

1. KafkaConsumer#poll(), since it returns offsets, will return nothing (or what KIP-288 suggests is to throw a TimeoutException while metadata is updating).

2. A ClientTimeoutException will be introduced to allow users to more clearly identify the reason why the method timed out. (e.g. LeaderNotAvailable, RequestTimeout etc)

...

Feasible and Rejected Alternatives

Please see KIP-288 for other rejected alternatives.

In discussion, many have raised the idea of using a new config to set timeout time for methods which is being changed in this KIP.  It would not be recommended to use one config for all methods. However, we could use it for similar methods (e.g. methods which call updateFetchPositions() will block using one timeout configured by the user). In this manner, we could incorporate both the config and the added timeout parameter into the code.

Another alternative of interest is that we should add a new overload for poll(), particularily since the changing the old method can become unwieldly between different Kafka versions. To elaborate, a Timeout parameter will also be added to the poll() overload.

One alternative was to add a timeout parameter to the current position() and other methods. However, the changes made by the user will be much more extensive then basing the time constraint on  requestTimeoutMs because the method signature has been changed. 

...