Versions Compared

Old Version 2

changes.mady.by.user TaiJuWu

Saved on

compared with

New Version 3

changes.mady.by.user TaiJuWu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Safety: Quorum checks and leader decisions are based on a membership configuration that is guaranteed to be replicated and agreed upon by a majority.

  • Stability: The committed voter set changes only at commit points, avoiding transient or rolled-back configurations.

  • Correctness under Reconfiguration: Both during joint consensus transitions and simple add/remove operations, the system preserves Raft’s safety guarantees.

Public Interfaces

Request Schema


Code Block
linenumberstrue
{
  "apiKey": 55,
  "type": "request",
  "listeners": ["broker", "controller"],
  "name": "DescribeQuorumRequest",
  // Version 1 adds additional fields in the response. The request is unchanged (KIP-836).
  // Version 2 adds additional fields in the response. The request is unchanged (KIP-853).
+ // Version 3 adds additional fields in the response. The request is unchanged.
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "latestVersionUnstable": false,
  "fields": [
    { "name": "Topics", "type": "[]TopicData", "versions": "0+",
      "about": "The topics to describe.", "fields": [
      { "name": "TopicName", "type": "string", "versions": "0+", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionData", "versions": "0+",
        "about": "The partitions to describe.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." }
      ]
      }]
    }
  ]
}

Response Schema


Code Block
linenumberstrue
{
  "apiKey": 55,
  "type": "response",
  "name": "DescribeQuorumResponse",
  // Version 1 adds LastFetchTimeStamp and LastCaughtUpTimestamp in ReplicaState (KIP-836).
  // Version 2 adds ErrorMessage, Nodes, ErrorMessage in ParitionData, ReplicaDirectoryId in ReplicaState (KIP-853).
+ // Version 3 adds CommittedVoters in PartitionData
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The top level error code."},
    { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
      "about": "The error message, or null if there was no error." },
    { "name": "Topics", "type": "[]TopicData",
      "versions": "0+", "fields": [
      { "name": "TopicName", "type": "string", "versions": "0+", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionData",
        "versions": "0+", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "ErrorCode", "type": "int16", "versions": "0+"},
        { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
          "about": "The error message, or null if there was no error." },
        { "name": "LeaderId", "type": "int32", "versions": "0+", "entityType": "brokerId",
          "about": "The ID of the current leader or -1 if the leader is unknown."},
        { "name": "LeaderEpoch", "type": "int32", "versions": "0+",
          "about": "The latest known leader epoch"},
        { "name": "HighWatermark", "type": "int64", "versions": "0+"},
        { "name": "CurrentVoters", "type": "[]ReplicaState", "versions": "0+" },
+       { "name": "CommittedVoters", "type": "[]ReplicaState", "versions": "3+", "nullableVersionsignorable": true "3+", "ignorableabout": true   "The voters has been committed."},
        { "name": "Observers", "type": "[]ReplicaState", "versions": "0+" }
      ]}
    ]},
    { "name": "Nodes", "type": "[]Node", "versions": "2+", "fields": [
      { "name": "NodeId", "type": "int32", "versions": "2+",
        "mapKey": true, "entityType": "brokerId", "about": "The ID of the associated node" },
      { "name": "Listeners", "type": "[]Listener",
        "about": "The listeners of this controller", "versions": "2+", "fields": [
        { "name": "Name", "type": "string", "versions": "2+", "mapKey": true,
          "about": "The name of the endpoint" },
        { "name": "Host", "type": "string", "versions": "2+",
          "about": "The hostname" },
        { "name": "Port", "type": "uint16", "versions": "2+",
          "about": "The port" }
      ]}
    ]}
  ],
  "commonStructs": [
    { "name": "ReplicaState", "versions": "0+", "fields": [
      { "name": "ReplicaId", "type": "int32", "versions": "0+", "entityType": "brokerId" },
      { "name": "ReplicaDirectoryId", "type": "uuid", "versions": "2+" },
      { "name": "LogEndOffset", "type": "int64", "versions": "0+",
        "about": "The last known log end offset of the follower or -1 if it is unknown"},
      { "name": "LastFetchTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        "about": "The last known leader wall clock time time when a follower fetched from the leader. This is reported as -1 both for the current leader or if it is unknown for a voter"},
      { "name": "LastCaughtUpTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        "about": "The leader wall clock append time of the offset for which the follower made the most recent fetch request. This is reported as the current time for the leader and -1 if unknown for a voter"}
    ]}
  ]
}

...

Code Block
languagejava
public class QuorumInfo {
    private final int leaderId;
    private final long leaderEpoch;
    private final long highWatermark;
    private final List<ReplicaState> voters;
+   private final List<ReplicaState> committedVoters;
    private final List<ReplicaState> observers;
    private final Map<Integer, Node> nodes;

    QuorumInfo(
        int leaderId,
        long leaderEpoch,
        long highWatermark,
        List<ReplicaState> voters,
        List<ReplicaState> committedVoters,
        List<ReplicaState> observers,
        Map<Integer, Node> nodes
    ) {
        this.leaderId = leaderId;
        this.leaderEpoch = leaderEpoch; 
        this.highWatermark = highWatermark;
        this.voters = voters;
+        this.committedVoters = committedVoters;
        this.observers = observers;
        this.nodes = nodes;
    }}

Proposed Changes

...

Raft State Tracking

This proposal introduces a new committed voter set tracking mechanism within the LeaderState to accurately represent the last committed quorum membership.

  • A new field committedVoterStates of type Map<Integer, ReplicaState> is added to the LeaderState class. This field stores the snapshot of the voter states that have been durably committed in the Raft log.

  • Within the maybeCommitToLeaderState() method (or equivalent commit processing logic), once the commit index advances and a quorum is confirmed for the new offset, the committedVoterStates map is updated to a copy of the current active currentVoterStates. This ensures the committed voter set reflects the durable state of membership at the last committed log entry.

  • The distinction between currentVoterStates (potentially changing during ongoing elections or reconfigurations) and committedVoterStates (stable, committed membership) enables clear separation of transient membership changes from those that are confirmed and durable.

  • An accessor method committedVoterStates() is provided on LeaderState to expose this snapshot. This method is used when constructing DescribeQuorumResponse messages to include the committed voter set, improving observability and operational diagnostics.

This mechanism strengthens the Raft membership model by making committed membership explicit and accessible to tooling and clients, thereby improving cluster state transparency.

Compatibility, Deprecation, and Migration Plan

  • Update DescribeQuorumRequest and DescribeQuorumResponse to a newer version, with backward compatibility preserved for the old protocol version.

...