Versions Compared

Old Version 2

changes.mady.by.user TaiJuWu

Saved on

compared with

New Version Current

changes.mady.by.user TaiJuWu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Under Discussion Draft

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

...

By maintaining a separate committedVoterSet, the system ensures:

  • Safety: Quorum checks and leader decisions are based on a membership configuration that is guaranteed to be replicated and agreed upon by a majority.

  • Stability: The committed voter set changes only at commit points, avoiding transient or rolled-back configurations.

  • Correctness under Reconfiguration: Both during joint consensus transitions and simple add/remove operations, the system preserves Raft’s safety guarantees.

Public Interfaces

Response Schema

  • Both committed and uncommitted voters participate in quorum decisions and elections. However, uncommitted voters (those whose VotersRecord has been appended but not yet committed) carry a risk: if the leader crashes before the VotersRecord is committed, the voter change maybe lost due to log truncation, leading to potential configuration inconsistencies.

  • Stability: When the committed and uncommitted voter sets differ, the cluster is in a joint consensus phase.
    Exposing this state allows management and orchestration systems to recognize that the controller quorum is undergoing a transitional reconfiguration, and to avoid assuming the new configuration is already stable.

  • Debugging and Compliance Verification: During troubleshooting or post-incident audits, operators and tools must be able to differentiate between committed voters and uncommitted voters that have started replicating but are not yet officially part of the quorum.This distinction is critical for confirming whether a reconfiguration completed successfully and whether the cluster’s control plane reached a consistent state.

Public Interfaces

Request Schema


Code Block
linenumberstrue
{
  "apiKey": 55,
  "type": "request",
  "listeners": ["broker", "controller"],
  "name": "DescribeQuorumRequest",
  // Version 1 adds additional fields in the response. The request is unchanged (KIP-836).
  // Version 2 adds additional fields in the response. The request is unchanged (KIP-853).
+ // Version 3 adds additional fields in the response. The request is unchanged.
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "latestVersionUnstable": false,
  "fields": [
    { "name": "Topics", "type": "[]TopicData", "versions": "0+"
Code Block
linenumberstrue
{
  "apiKey": 55,
  "type": "response",
  "name": "DescribeQuorumResponse",
  // Version 1 adds LastFetchTimeStamp and LastCaughtUpTimestamp in ReplicaState (KIP-836).
  // Version 2 adds ErrorMessage, Nodes, ErrorMessage in ParitionData, ReplicaDirectoryId in ReplicaState (KIP-853).
+ // Version 3 adds CommittedVoters in PartitionData
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The top level error code."},
    { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
      "about": "The error message, or null if there was no error." },
    topics to describe.", "fields": [
      { "name": "TopicsTopicName", "type": "[]TopicDatastring",
      "versions": "0+", "fieldsentityType": ["topicName",
      {  "nameabout": "TopicName", "type": "stringThe topic name." },
      { "name": "Partitions", "versionstype": "0+[]PartitionData", "entityTypeversions": "topicName0+",
        "about": "The topicpartitions to namedescribe.", "fields": },[
        { "name": "PartitionsPartitionIndex", "type": "[]PartitionDataint32",
        "versions": "0+",
 "fields": [
        { "nameabout": "PartitionIndex", "type": "int32", "versions": "0+",The partition index." }
        ]
  "about": "The partition index." },]
    }
  ]
}

Response Schema


Code Block
linenumberstrue
{
  "apiKey  { "name": "ErrorCode"55,
  "type": "int16response",
  "versionsname": "0+DescribeQuorumResponse"},
  // Version 1 adds LastFetchTimeStamp and { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
          "about": "The error message, or null if there was no error." },
    LastCaughtUpTimestamp in ReplicaState (KIP-836).
  // Version 2 adds ErrorMessage, Nodes, ErrorMessage in ParitionData, ReplicaDirectoryId in ReplicaState (KIP-853).
+ // Version 3 adds CommittedVoters in PartitionData
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "LeaderIdErrorCode", "type": "int32int16", "versions": "0+",
 "entityType": "brokerId",
            "about": "The IDtop oflevel the current leader or -1 if the leader is unknownerror code."},
        { "name": "LeaderEpochErrorMessage", "type": "int32string", "versions": "02+",
 "nullableVersions": "2+", "ignorable": true,
      "about": "The latest known leader epoch"error message, or null if there was no error." },
        { "name": "HighWatermarkTopics", "type": "int64[]TopicData",
      "versions": "0+"},
 "fields": [
      { "name": "CurrentVotersTopicName", "type": "[]ReplicaStatestring", "versions": "0+", "entityType": "topicName",
        "about": "The topic name." },
+       { "name": "CommittedVotersPartitions", "type": "[]ReplicaStatePartitionData",
        "versions": "30+", "nullableVersionsfields": "3+", "ignorable": true },[
        { "name": "ObserversPartitionIndex", "type": "[]ReplicaStateint32", "versions": "0+" }
,
          ]}
    ]"about": "The partition index." },
        { "name": "NodesErrorCode", "type": "[]Nodeint16", "versions": "20+"}, "fields": [

        { "name": "NodeIdErrorMessage", "type": "int32string", "versions": "2+",
        "mapKey"nullableVersions": true, "entityType": "brokerId"2+", "aboutignorable": "The ID of the associated node" },
true,
         { "nameabout": "Listeners", "type": "[]Listener"The error message, or null if there was no error." },
        { "aboutname": "LeaderId"The listeners of this controller, "type": "int32", "versions": "20+", "fieldsentityType": ["brokerId",
        {  "nameabout": "Name", "typeThe ID of the current leader or -1 if the leader is unknown."},
        { "name": "stringLeaderEpoch", "versionstype": "2+int32", "mapKeyversions": true"0+",
          "about": "The namelatest ofknown theleader endpointepoch" },
        { "name": "HostHighWatermark", "type": "stringint64", "versions": "20+"},
        { "name": "CurrentVoters", "abouttype": "The hostname[]ReplicaState", "versions": "0+" },
 +       { "name": "PortCommittedVoters", "type": "uint16[]ReplicaState", "versions": "23+",
  "ignorable": true   ,    "about":  "The port"voters }
has      ]}been committed."},
    ]}
   ],
 { "commonStructsname": [
    { "name"Observers", "type": "[]ReplicaState", "versions": "0+", "fields": [ }
      ]}
    ]},
    { "name": "ReplicaIdNodes", "type": "int32[]Node", "versions": "02+", "entityTypefields": "brokerId" },[
      { "name": "ReplicaDirectoryIdNodeId", "type": "uuidint32", "versions": "2+" },
      {  "namemapKey": "LogEndOffset"true, "typeentityType": "int64brokerId", "versionsabout": "0+",
        "about": "The last known log end offset The ID of the follower or -1 if it is unknown"associated node" },
      { "name": "LastFetchTimestampListeners", "type": "int64[]Listener",
  "versions": "1+      "about": "The listeners of this controller", "ignorableversions": true"2+", "defaultfields": -1,[
        { "aboutname": "Name"The last known leader wall clock time time when a follower fetched from the leader. This is reported as -1 both for the current leader or if it is unknown for a voter"},
      { ", "type": "string", "versions": "2+", "mapKey": true,
          "about": "The name of the endpoint" },
        { "name": "LastCaughtUpTimestampHost", "type": "int64string", "versions": "12+",
 "ignorable": true, "default": -1,
        "about": "The leader wall clock append time of the offset for which the follower made the most recent fetch request. This is reported as the current time for the leader and -1 if unknown for a voter"hostname" },
        { "name": "Port", "type": "uint16", "versions": "2+",
          "about": "The port" }
      ]}
    ]}
  ]
}

We also add new field committedVoters to QuorumInfo.

Code Block
languagejava
public class QuorumInfo {,
  "commonStructs": [
    private final int leaderId;
    private final long leaderEpoch;{ "name": "ReplicaState", "versions": "0+", "fields": [
    private final long highWatermark;
    private final List<ReplicaState> voters;
+   private final List<ReplicaState> committedVoters;
    private final List<ReplicaState> observers;{ "name": "ReplicaId", "type": "int32", "versions": "0+", "entityType": "brokerId" },
      { "name": "ReplicaDirectoryId", "type": "uuid", "versions": "2+" },
    private final Map<Integer, Node> nodes;

    QuorumInfo({ "name": "LogEndOffset", "type": "int64", "versions": "0+",
        int leaderId,
        long leaderEpoch,
        long highWatermark,
        List<ReplicaState> voters,
        List<ReplicaState> committedVoters"about": "The last known log end offset of the follower or -1 if it is unknown"},
      { "name": "LastFetchTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        List<ReplicaState> observers,
        Map<Integer, Node> nodes
    ) {
        this.leaderId = leaderId;
        this.leaderEpoch = leaderEpoch; 
        this.highWatermark = highWatermark;"about": "The last known leader wall clock time time when a follower fetched from the leader. This is reported as -1 both for the current leader or if it is unknown for a voter"},
      { "name": "LastCaughtUpTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        this.voters = voters;
        this.committedVoters = committedVoters;
        this.observers = observers;
        this.nodes = nodes;
    }}

Proposed Changes

...

"about": "The leader wall clock append time of the offset for which the follower made the most recent fetch request. This is reported as the current time for the leader and -1 if unknown for a voter"}
    ]}
  ]
}

We also add new field committedVoters to QuorumInfo.


Code Block
languagejava
public class QuorumInfo {
    private final int leaderId;
    private final long leaderEpoch;
    private final long highWatermark;
    private final List<ReplicaState> voters;
+   private final List<ReplicaState> committedVoters;
    private final List<ReplicaState> observers;
    private final Map<Integer, Node> nodes;

    QuorumInfo(
        int leaderId,
        long leaderEpoch,
        long highWatermark,
        List<ReplicaState> voters,
        List<ReplicaState> committedVoters,
        List<ReplicaState> observers,
        Map<Integer, Node> nodes
    ) {
        this.leaderId = leaderId;
        this.leaderEpoch = leaderEpoch; 
        this.highWatermark = highWatermark;
        this.voters = voters;
+       this.committedVoters = committedVoters;
        this.observers = observers;
        this.nodes = nodes;
    }}

Proposed Changes

Raft State Tracking

This proposal introduces a mechanism for explicit tracking the committed voter set within LeaderState. The objective is to provide an accurate representation of the last committed quorum membership.

The leaderState is extended to reference the KRaftControlRecordStateMachine, from which derives the voter set history. Using this information the of high watermark from LeaderState, the committed voter set can be obtained directly from KRaftControlRecordStateMachine. Both the voter set in the KRaftControlRecordStateMachine and the high watermark are maintained entirely in memory. This eliminates disk I/O and minimizing performance impact.

Committed voter present in current voters/observers
If a committed voter is also present in the current voters or observers set, its ReplicaState is identical to the corresponding entry in those sets. This ensures that all runtime fields within ReplicaState are available and reflect the current replica state.

Committed voter absent from voters or observers
If a committed voter is not found in either voters or observers, certain runtime-specific fields cannot be derived, since the replica is no longer actively participating in log replication. To maintain schema consistency, these fields in its ReplicaState are set to sentinel values:

  • hasAcknowledgedLeader = false

  • endLogOffset = -1

  • lastFetchTimestamp = -1

  • lastCaughtUpTimestamp = -1

Above fields are only meaningful for active replicas(voter) in the quorum. For committed voters, they are not strictly required—the sentinel values simply indicate that the data is unavailable or not applicable.

High watermark is unknown when a new leader is elected 

  • In such case, the empty set should be returned because we don't know the high watermark.
  • Committed voters can be returned as non-empty when leader update the high watermark

...

  • .

Compatibility, Deprecation, and Migration Plan

  • Update Bump DescribeQuorumRequest and DescribeQuorumResponse to a newer version, with backward compatibility preserved for the old protocol version.

Test Plan

  • Unit test and integration test will be added.

Rejected Alternatives

...

  • A new field, state, can be introduced within the ReplicaState structure to explicitly track the status of the voter like uncommitted and committed.