THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Table of Contents |
|---|
This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Status
Current state: WIP
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Note this is a joint proposal by Philip Nee, Kirk True, and Lianet Magrans.
Motivation
This KIP documents the updated threading model of the Consumer implementation of the client.
...
To help understand the design, we need to introduce some terminology. Terms designated with 1 apply to the current KafkaConsumer implementation and terms with 2 apply to the new implementation; a term may apply to both.
| Term | Definition |
|---|---|
Application event | A data structure specific to each Application |
...
events can optionally include a | |
Application event processor | Logic which processes application events on the background thread, interacting with the request managers. |
Application |
...
event queue | A shared queue which stores application events enqueued by the application thread. These events are later dequeued by the background thread and given to the application event processor for execution. |
Application thread | The thread that is executing the user's code that interacts with the |
| Background |
...
| event | |
| Background event queue | A shared queue which stores background events enqueued by the background thread. The events are later dequeued by the application thread inside the |
| Background thread | An internal thread |
...
created for each |
...
instance on which the following operations are performed:
|
| Event handler |
| Logic that pulls events from the event queue for processing on the background thread. |
...
| Heartbeat | Logic related to communicating liveness, group membership, etc. as introduced in KIP-62. |
| Network client delegate |
...
| Request manager |
...
| An internal interface that is used by the background thread to handle the management of requested, inflight, and responded network I/O. |
Threading Model
<TBD>
Background thread
...