Page History

...

If the Consumer API is a blocking call, the event passed from the application thread to the network thread will include an embedded CompleteableFuture. After enqueuing the event, the application thread will invoke Future.get(), effectively blocking itself until a result is provided. When the result for the Consumer API call is ready, the network thread will then invoke CompeteableFuture.complete() with the result, allowing the application thread to continue execution.

...

Submitting Client Requests

The following diagram displays the basic flow between the request managers, unsent request queue, and the NetworkClient: 

Request managers collect together the logic needed to issue the different client RPC requests and then handle their responses accordingly.

In the network thread, we loop over each request manager, effectively asking it for any requests that it needs to send to the cluster. Note that during this step the network request is not sent. Instead, an unsent request object is created which contains the underlying request information. These "unsent requests" are added to a queue of pending unsent client requests. After all of these unsent requests are queued up, then they are forwarded for network I/O via the NetworkClient.send() method.

There are two benefits for this multi-step process:

1. It keeps the network I/O request and response management and lifecycle in one place, making the code easier to reason about
2. The request managers can implement deduplication and/or coalescing of requests

Terminologies:

• CB: Callbacks registered and invoked on the polling thread: commit callback, rebalance callback.
• rm: RequestManagers. e.g. Heartbeat, FindCoordinatorRequest.
• subscriptionState design is still under discussion

...