DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: VotingAccepted
Discussion thread: here
Vote thread: here
...
The start offset is regularly updated and persisted through the writeShareGroupState RPC call in the __share_group_state topic, managed by the Share Coordinator. This KIP introduces a mechanism to calculate and persist the count of in-flight records that have already been processed, and makes it available to the users.
...
.
For each share partition, the leader will compute this new information and include it in the writeShareGroupState RPC, which will be persisted via the ShareSnapshot and ShareUpdate records in the __share_group_state topic. The same information will also be returned in the response of ReadShareGroupStateSummary.
To make this information available to users, the Group Coordinator retrieves it through the ReadShareGroupStateSummary API and computes the share-partition lag, which is then included in the response to the DescribeShareGroupOffsets request invoked by Admin.listShareGroupOffsets(). To calculate the lag, the Group Coordinator issues an Admin.listOffsets() call to fetch the end offset of the underlying partition and then applies the share-partition lag formula defined above to derive the lag value.
| Info |
|---|
The SPEO is intentionally excluded from both the external interfaces and the share partition lag calculations, since future changes may allow sparse in-flight records, and the distance between the SPSO and the SPEO can vary significantly. The concept of lag introduced in this KIP is therefore designed to remain flexible and extensible to accommodate such future evolutions. |
Public Interfaces
Client API changes
...