Page History

...

• Application thread: user thread on which all Consumer API requests and callback handlers are executed.

• Network I/O thread: client-internal thread on which network I/O related to heartbeat, coordinator, and rebalance is executed.

• Event Handlerhandler: A communication interface between the application thread and network thread

• Application event queue: sends events to the network I/O thread from the application thread

• Background event queue: sends events from the network thread to the application thread

We will discuss the strategy for handling event results in the following section. In short, for async APIs like commitAsync(), CompletableFuture will be used to notify the completion of the events.  

Application and Network Thread

The following diagram depicts the interaction between the application thread and the network thread:

|

Image Removed

Terminologies:

The Consumer client object is here depicted in purple. In this design, instead of directly operating on the parameters given to the various APIs (subscribe(), poll(), commit(), etc.), the Consumer implementation packages the parameters as events that are enqueued on the application event queue.

The event handler, shown in green, executes on the network thread. In each loop of the network thread, events are read from the queue and processed.

If the Consumer API is a blocking call, the event passed from the application thread to the network thread will include an embedded CompleteableFuture. After enqueuing the event, the application thread will invoke Future.get(), effectively blocking itself until a result is provided. When the result for the Consumer API call is ready, the network thread will then invoke CompeteableFuture.complete() with the result, allowing the application thread to continue execution.

Image Added

Terminologies:

• CB: Callbacks registered and invoked on the polling thread: CB: Callbacks registered and invoked on the polling thread: commit callback, rebalance callback.
• rm: RequestManagers. e.g. Heartbeat, FindCoordinatorRequest.
• subscriptionState design is still under discussion

...

Polling thread and its lifecycle

...