...

• Deprecate HeartbeatThread

  • Remove the heartbeat thread.
  • Move the heartbeat logic to the background thread.
• Implement a background thread

• Refactor the Coordinator classes

  • Revisit timers and loops
  • Blocking methods such as commitOffsetSync will be handled by the poling thread by waiting on the future's completion. The coordinator should not be blocking.
  • The background thread loop will poll the coordinator.
  • We will modify the existing rebalance states, specifically for the callback execution.

• Refactor the KafkaConsumer API

  • It will send Events to the background thread if network communication is required.
  • Remove dependency on the fetcher, coordinator, and networkClient.
• Minor Refactor:
  • Fetcher:
    • sendFetches will be performed in the background thread.
    • network client, coordinator will be removed from the class

• Events and communication channels.

  • We will use two channels the facilitate BlockingQueues for the two-way communication between the background thread and the polling thread.

• Address issues in these Jira tickets

...

• The object is frequently accessed by both the polling thread and background thread.
• The object often requires instaneous response; otherwise, the performance will be heavily impacted
• Also, please review the rejected proposals section below. I talked about a few ideas that we've had.

Image RemovedImage Added

Important Components

...