Versions Compared

Old Version 1

changes.mady.by.user TaiJuWu

Saved on

compared with

New Version 2

changes.mady.by.user TaiJuWu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In a Raft-based quorum system, cluster membership changes (adding or removing voters) are applied through the log replication process. A membership change becomes effective only after it has been committed — that is, replicated to a majority of the current configuration. Between the moment a membership change entry is appended and the moment it is committed, the cluster exists in a transitional state where the leader’s in-memory voter list (voterSet) may include uncommitted changes.

By maintaining a separate committedVoterSet, the system ensures:

  • Safety: Quorum checks and leader decisions are based on a membership configuration that is guaranteed to be replicated and agreed upon by a majority.

  • Stability: The committed voter set changes only at commit points, avoiding transient or rolled-back configurations.

  • Correctness under Reconfiguration: Both during joint consensus transitions and simple add/remove operations, the system preserves Raft’s safety guarantees.

Public Interfaces

...

Response Schema


Code Block
linenumberstrue
{
  "apiKey": 55,
  "type": "response",
  "name": "DescribeQuorumResponse",
  // Version 1 adds LastFetchTimeStamp and LastCaughtUpTimestamp in ReplicaState (KIP-836).
  // Version 2 adds ErrorMessage, Nodes, ErrorMessage in ParitionData, ReplicaDirectoryId in ReplicaState (KIP-853).
+ // Version 3 adds CommittedVoters in PartitionData
+ "validVersions": "0-3",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The top level error code."},
    { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
      "about": "The error message, or null if there was no error." },
    { "name": "Topics", "type": "[]TopicData",
      "versions": "0+", "fields": [
      { "name": "TopicName", "type": "string", "versions": "0+", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionData",
        "versions": "0+", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "ErrorCode", "type": "int16", "versions": "0+"},
        { "name": "ErrorMessage", "type": "string", "versions": "2+", "nullableVersions": "2+", "ignorable": true,
          "about": "The error message, or null if there was no error." },
        { "name": "LeaderId", "type": "int32", "versions": "0+", "entityType": "brokerId",
          "about": "The ID of the current leader or -1 if the leader is unknown."},
        { "name": "LeaderEpoch", "type": "int32", "versions": "0+",
          "about": "The latest known leader epoch"},
        { "name": "HighWatermark", "type": "int64", "versions": "0+"},
        { "name": "CurrentVoters", "type": "[]ReplicaState", "versions": "0+" },
+       { "name": "CommittedVoters", "type": "[]ReplicaState", "versions": "3+", "nullableVersions": "3+", "ignorable": true },
        { "name": "Observers", "type": "[]ReplicaState", "versions": "0+" }
      ]}
    ]},
    { "name": "Nodes", "type": "[]Node", "versions": "2+", "fields": [
      { "name": "NodeId", "type": "int32", "versions": "2+",
        "mapKey": true, "entityType": "brokerId", "about": "The ID of the associated node" },
      { "name": "Listeners", "type": "[]Listener",
        "about": "The listeners of this controller", "versions": "2+", "fields": [
        { "name": "Name", "type": "string", "versions": "2+", "mapKey": true,
          "about": "The name of the endpoint" },
        { "name": "Host", "type": "string", "versions": "2+",
          "about": "The hostname" },
        { "name": "Port", "type": "uint16", "versions": "2+",
          "about": "The port" }
      ]}
    ]}
  ],
  "commonStructs": [
    { "name": "ReplicaState", "versions": "0+", "fields": [
      { "name": "ReplicaId", "type": "int32", "versions": "0+", "entityType": "brokerId" },
      { "name": "ReplicaDirectoryId", "type": "uuid", "versions": "2+" },
      { "name": "LogEndOffset", "type": "int64", "versions": "0+",
        "about": "The last known log end offset of the follower or -1 if it is unknown"},
      { "name": "LastFetchTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        "about": "The last known leader wall clock time time when a follower fetched from the leader. This is reported as -1 both for the current leader or if it is unknown for a voter"},
      { "name": "LastCaughtUpTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
        "about": "The leader wall clock append time of the offset for which the follower made the most recent fetch request. This is reported as the current time for the leader and -1 if unknown for a voter"}
    ]}
  ]
}

...

Proposed Changes

Update the RaftClient to record the committedVoterSet whenever the high watermark advances. This ensures that the stored voter set always reflects the voters from the latest committed configuration, avoiding inconsistencies that could arise from uncommitted membership changes.

Compatibility, Deprecation, and Migration Plan

  • Update DescribeQuorumResponse to a newer version, with backward compatibility preserved for the old protocol version.

...