Current state: Under Discussion
Discussion thread: TODO
JIRA:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
The FETCH RPC is used to replicate both the MemoryRecords and high-watermark (HWM) for the partition. The FETCH response contains the latest HWM known by the leader and any bytes written to the partition after the FetchOffset in the FETCH request. In KRaft all of the replicas fetch uncommitted data. When the FetchOffset in the FETCH request is equal to the log end offset (LEO), KRaft parks those requests until new data is available. Once the active controller appends new records, KRaft completes all of the parked requests with the new appended records.
KRaft is not able to complete FETCH RPC based on HWM because the KRaft leader doesn't know what HWM was replicated to the remove replicas. Because of this, if the HWM changes but no new data is append HWM replica can be delayed by at most 500ms. 500ms is the maximum amount of time that the KRaft leader will park FETCH requests if there are no records after the FetchOffset.
This KIP proposes adding the replica's HWM to the FETCH request so that the KRaft leader can complete any FETCH request in which the current HWM doesn't match the replica's HWM.
Add a version 18 of the RPC. Add the field HighWatermark which the replica sets to is known high-watermark. Set to -1 if the high-watermark is not known (TODO: check that kraft users -1 on the leader when the high-watermark is not known).
{
"apiKey": 1,
"type": "request",
"listeners": ["broker", "controller"],
"name": "FetchRequest",
"validVersions": "4-18",
"flexibleVersions": "12+",
"fields": [
{ "name": "ClusterId", "type": "string", "versions": "12+", "nullableVersions": "12+", "default": "null",
"taggedVersions": "12+", "tag": 0, "ignorable": true,
"about": "The clusterId if known. This is used to validate metadata fetches prior to broker registration." },
{ "name": "ReplicaId", "type": "int32", "versions": "0-14", "default": "-1", "entityType": "brokerId",
"about": "The broker ID of the follower, of -1 if this request is from a consumer." },
{ "name": "ReplicaState", "type": "ReplicaState", "versions": "15+", "taggedVersions": "15+", "tag": 1,
"about": "The state of the replica in the follower.", "fields": [
{ "name": "ReplicaId", "type": "int32", "versions": "15+", "default": "-1", "entityType": "brokerId",
"about": "The replica ID of the follower, or -1 if this request is from a consumer." },
{ "name": "ReplicaEpoch", "type": "int64", "versions": "15+", "default": "-1",
"about": "The epoch of this follower, or -1 if not available." }
]},
{ "name": "MaxWaitMs", "type": "int32", "versions": "0+",
"about": "The maximum time in milliseconds to wait for the response." },
{ "name": "MinBytes", "type": "int32", "versions": "0+",
"about": "The minimum bytes to accumulate in the response." },
{ "name": "MaxBytes", "type": "int32", "versions": "3+", "default": "0x7fffffff", "ignorable": true,
"about": "The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored." },
{ "name": "IsolationLevel", "type": "int8", "versions": "4+", "default": "0", "ignorable": true,
"about": "This setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to discard ABORTED transactional records." },
{ "name": "SessionId", "type": "int32", "versions": "7+", "default": "0", "ignorable": true,
"about": "The fetch session ID." },
{ "name": "SessionEpoch", "type": "int32", "versions": "7+", "default": "-1", "ignorable": true,
"about": "The fetch session epoch, which is used for ordering requests in a session." },
{ "name": "Topics", "type": "[]FetchTopic", "versions": "0+",
"about": "The topics to fetch.", "fields": [
{ "name": "Topic", "type": "string", "versions": "0-12", "entityType": "topicName", "ignorable": true,
"about": "The name of the topic to fetch." },
{ "name": "TopicId", "type": "uuid", "versions": "13+", "ignorable": true, "about": "The unique topic ID."},
{ "name": "Partitions", "type": "[]FetchPartition", "versions": "0+",
"about": "The partitions to fetch.", "fields": [
{ "name": "Partition", "type": "int32", "versions": "0+",
"about": "The partition index." },
{ "name": "CurrentLeaderEpoch", "type": "int32", "versions": "9+", "default": "-1", "ignorable": true,
"about": "The current leader epoch of the partition." },
{ "name": "FetchOffset", "type": "int64", "versions": "0+",
"about": "The message offset." },
{ "name": "LastFetchedEpoch", "type": "int32", "versions": "12+", "default": "-1", "ignorable": false,
"about": "The epoch of the last fetched record or -1 if there is none."},
{ "name": "LogStartOffset", "type": "int64", "versions": "5+", "default": "-1", "ignorable": true,
"about": "The earliest available offset of the follower replica. The field is only used when the request is sent by the follower."},
{ "name": "PartitionMaxBytes", "type": "int32", "versions": "0+",
"about": "The maximum bytes to fetch from this partition. See KIP-74 for cases where this limit may not be honored." },
{ "name": "ReplicaDirectoryId", "type": "uuid", "versions": "17+", "taggedVersions": "17+", "tag": 0, "ignorable": true,
"about": "The directory id of the follower fetching." },
{ "name": "HighWatermark", "type": "int64", "versions": "18+", "default": "9223372036854775807", "taggedVersions": 18+", "tag": 1, "ignorable": true,
"about": "The high-watermark known by the replica." }
]}
]},
{ "name": "ForgottenTopicsData", "type": "[]ForgottenTopic", "versions": "7+", "ignorable": false,
"about": "In an incremental fetch request, the partitions to remove.", "fields": [
{ "name": "Topic", "type": "string", "versions": "7-12", "entityType": "topicName", "ignorable": true,
"about": "The topic name." },
{ "name": "TopicId", "type": "uuid", "versions": "13+", "ignorable": true, "about": "The unique topic ID."},
{ "name": "Partitions", "type": "[]int32", "versions": "7+",
"about": "The partitions indexes to forget." }
]},
{ "name": "RackId", "type": "string", "versions": "11+", "default": "", "ignorable": true,
"about": "Rack ID of the consumer making this request."}
]
} |
No schema changes to the response. The version will be increase to 18 to match the request version.
TODO
TODO
TODO
Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.