THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- Fail the request of the voter ID and UUID doesn't match the local ID and UUID.
- All replicas that can become voter are allowed to vote even if they are not voters yet for the quorum.
- If granted persist the Candidate GUID and ID.
BeginQuorumEpoch
Request
- LeaderId was moved out of the topic partition maps
- VoterId was added to the top level
- VoterUuId was added to PartitionData
- Allow tagged fields for version greater than or equal to 1.
| Code Block |
|---|
{
"apiKey": 53,
"type": "request",
"listeners": ["controller"],
"name": "BeginQuorumEpochRequest",
"validVersions": "0-1",
"flexibleVersions": "none1+",
"fields": [
{ "name": "ClusterId", "type": "string", "versions": "0+",
"nullableVersions": "0+", "default": "null"},
{ "name": "TopicsLeaderId", "type": "[]TopicDataint32", "versions": "01+", "fieldsentityType": [
"brokerId",
"about": "The ID of the newly elected leader"},
{ "name": "TopicNameVoterId", "type": "stringint32", "versions": "01+", "entityType": "topicNamebrokerId",
"about": "The topic namevoter ID of the receiving replica." },
{ "name": "PartitionsTopics", "type": "[]PartitionDataTopicData", "versions": "0+", "fields": [
{ "name": "PartitionIndexTopicName", "type": "int32string", "versions": "0+",
"entityType": "topicName",
"about": "The partitiontopic indexname." },
{ "name": "LeaderIdPartitions", "type": "int32[]PartitionData", "versions": "0+", "entityTypefields": "brokerId",
"about": "The ID of the newly elected leader"},
[
{ "name": "LeaderEpochPartitionIndex", "type": "int32", "versions": "0+",
"about": "The epochpartition of the newly elected leader"}
]index." },
{ "name": "VoterUuid", "type": "uuid", "versions": "1+",
"about": "The voter UUID of the receiving replica." },
{ "name": "LeaderId", "type": "int32", "versions": "0", "entityType": "brokerId",
"about": "The ID of the newly elected leader"},
{ "name": "LeaderEpoch", "type": "int32", "versions": "0+",
"about": "The epoch of the newly elected leader"}
]}
]}
]
} |
Handling
- Reject if the voter ID and UUID doesn't match.
- Reject if receiving voter hasn't read the AddVoterRecord with its voter id and UUID. May need to add a new error code. The leader will retry this error type.
- Handle as normal
EndQuorumEpoch
Request
- LeaderId was moved out of the topic partition maps
- VoterId was added to the request
- VoterUuid was added to the Partitions
- ReplicaUuid was added to PreferredSuccessors
- Allow tagged fields for versions greater than or equal to 1.
- TODO: Verify that the changes to PreferredSuccessors are backwards compatible.
| Code Block |
|---|
{
"apiKey": 54,
"type": "request",
"listeners": ["controller"],
"name": "EndQuorumEpochRequest",
"validVersions": "0-1",
"flexibleVersions": "1+",
"fields": [
{ "name": "ClusterId", "type": "string", "versions": "0+",
"nullableVersions": "0+", "default": "null"},
{ "name": "LeaderId", "type": "int32", "versions": "1+", "entityType": "brokerId",
"about": "The current leader ID that is resigning." },
{ "name": "VoterId", "type": "int32", "versions": "1+", "entityType": "brokerId",
"about": "The voter ID of the receiving replica." },
{ "name": "Topics", "type": "[]TopicData", "versions": "0+", "fields": [
{ "name": "TopicName", "type": "string", "versions": "0+", "entityType": "topicName",
"about": "The topic name." },
{ "name": "Partitions", "type": "[]PartitionData", "versions": "0+", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0+",
"about": "The partition index." },
{ "name": "VoterUuid", "type": "uuid", "versions": "1+",
"about": "The voter UUID of the receiving replica." },
{ "name": "LeaderId", "type": "int32", "versions": "0", "entityType": "brokerId",
"about": "The current leader ID that is resigning"},
{ "name": "LeaderEpoch", "type": "int32", "versions": "0+",
"about": "The current epoch"},
{ "name": "PreferredSuccessors", "type": "[]ReplicaInfo", "versions": "0+",
"about": "A sorted list of preferred successors to start the election", "fields": [
{ "name": "ReplicaId", "type": "int32", "versions": "0+", "entityType": "brokerId" },
{ "name": "ReplicaUuid", "type": "uuid", "versions": "1+" }
]}
]}
]}
]
} |
DescribeQuorum
The version is of the request is increase and the fields remain unchanged.
Response
- Add ReplicaUuid to ReplicaState
| Code Block |
|---|
{
"apiKey": 55,
"type": "response",
"name": "DescribeQuorumResponse",
// Version 1 adds LastFetchTimeStamp and LastCaughtUpTimestamp in ReplicaState (KIP-836).
"validVersions": "0-1",
"flexibleVersions": "0+",
"fields": [
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The top level error code."},
{ "name": "Topics", "type": "[]TopicData",
"versions": "0+", "fields": [
{ "name": "TopicName", "type": "string", "versions": "0+", "entityType": "topicName",
"about": "The topic name." },
{ "name": "Partitions", "type": "[]PartitionData",
"versions": "0+", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0+",
"about": "The partition index." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+"},
{ "name": "LeaderId", "type": "int32", "versions": "0+", "entityType": "brokerId",
"about": "The ID of the current leader or -1 if the leader is unknown."},
{ "name": "LeaderEpoch", "type": "int32", "versions": "0+",
"about": "The latest known leader epoch"},
{ "name": "HighWatermark", "type": "int64", "versions": "0+"},
{ "name": "CurrentVoters", "type": "[]ReplicaState", "versions": "0+" },
{ "name": "Observers", "type": "[]ReplicaState", "versions": "0+" }
]}
]}
],
"commonStructs": [
{ "name": "ReplicaState", "versions": "0+", "fields": [
{ "name": "ReplicaId", "type": "int32", "versions": "0+", "entityType": "brokerId" },
{ "name": "ReplicaUuid", "type": "uuid", "versions": "1+" },
{ "name": "LogEndOffset", "type": "int64", "versions": "0+",
"about": "The last known log end offset of the follower or -1 if it is unknown"},
{ "name": "LastFetchTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
"about": "The last known leader wall clock time time when a follower fetched from the leader. This is reported as -1 both for the current leader or if it is unknown for a voter"},
{ "name": "LastCaughtUpTimestamp", "type": "int64", "versions": "1+", "ignorable": true, "default": -1,
"about": "The leader wall clock append time of the offset for which the follower made the
most recent fetch request. This is reported as the current time for the leader and -1 if unknown for a voter" }
]}
]
} |
DescribeQuorum
The version is of the request is increase and the fields remain unchanged.
Response
- Replica GUID UUID in ReplicaState
- Leader needs to distinguish between replicas that are fetching as observer but should become voters vs replicas that are observers but cannot become voters.
Handling
- Leader needs to include all replicas ID and GUID UUID in the quorum
- CLI CLI - kafka-quorum
- Kafka needs a CLI that can display this information.
- The CLI can assume that if a replica has both the Id and UUID then it can be become a voter.
Admin Client
TODO: Document operations for adding voter and removing voters
...