THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Required ACL
- Describe Group
Request Validation
No particular changes.
Request Handling
The new types filter is handled.
Response Schema
The GroupType field is introduced. It represents the type of the group.
| Code Block | ||
|---|---|---|
| ||
{
"apiKey": 16,
"type": "response",
"name": "ListGroupsResponse",
// Version 1 adds the throttle time.
//
// Starting in version 2, on quota violation, brokers send out
// responses before throttling.
//
// Version 3 is the first flexible version.
//
// Version 4 adds the GroupState field (KIP-518).
//
// Version 5 adds the GroupType field (KIP-848).
"validVersions": "0-5",
"flexibleVersions": "3+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "1+",
"ignorable": true,
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "Groups", "type": "[]ListedGroup", "versions": "0+",
"about": "Each group in the response.", "fields": [
{ "name": "GroupId", "type": "string", "versions": "0+",
"entityType": "groupId",
"about": "The group ID." },
{ "name": "ProtocolType", "type": "string", "versions": "0+",
"about": "The group protocol type." },
{ "name": "GroupState", "type": "string", "versions": "4+", "ignorable": true,
"about": "The group state name." },
{ "name": "GroupType", "type": "string", "versions": "5+", "ignorable": true,
"about": "The group state name." }
]}
]
} |
Response Handling
No changes.
OffsetCommit API
The version of the API is bumped to 9 to add support for topic ids. The request can either use topic ids or topic names. The consumer will only use topic ids when they are available whereas the admin client will continue to use topic names as per its API.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"apiKey": 8,
"type": "request",
"listeners": ["zkBroker", "broker"],
"name": "OffsetCommitRequest",
// Version 1 adds timestamp and group membership information, as well as the commit timestamp.
//
// Version 2 adds retention time. It removes the commit timestamp added in version 1.
//
// Version 3 and 4 are the same as version 2.
//
// Version 5 removes the retention time, which is now controlled only by a broker configuration.
//
// Version 6 adds the leader epoch for fencing.
//
// version 7 adds a new field called groupInstanceId to indicate member identity across restarts.
//
// Version 8 is the first flexible version.
//
// Version 9 adds TopicId field (KIP-848).
"validVersions": "0-9",
"flexibleVersions": "8+",
"fields": [
{ "name": "GroupId", "type": "string", "versions": "0+", "entityType": "groupId",
"about": "The unique group identifier." },
// Renamed field.
{ "name": "GenerationIdOrMemberEpoch", "type": "int32", "versions": "1+", "default": "-1", "ignorable": true,
"about": "The generation of the group if the generic group protocol or the member epoch if the consumer protocol." },
{ "name": "MemberId", "type": "string", "versions": "1+", "ignorable": true,
"about": "The member ID assigned by the group coordinator." },
{ "name": "GroupInstanceId", "type": "string", "versions": "7+",
"nullableVersions": "7+", "default": "null",
"about": "The unique identifier of the consumer instance provided by end user." },
{ "name": "RetentionTimeMs", "type": "int64", "versions": "2-4", "default": "-1", "ignorable": true,
"about": "The time period in ms to retain the offset." },
{ "name": "Topics", "type": "[]OffsetCommitRequestTopic", "versions": "0+",
"about": "The topics to commit offsets for.", "fields": [
// Updated field.
{ "name": "Name", "type": "string", "versions": "0+", "nullableVersions": "9+", "default": "null", "entityType": "topicName",
"about": "The topic name."},
// New field.
{ "name": "TopicId", "type": "uuid", "versions": "9+", "about": "The unique topic ID" },
{ "name": "Partitions", "type": "[]OffsetCommitRequestPartition", "versions": "0+",
"about": "Each partition to commit offsets for.", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0+",
"about": "The partition index." },
{ "name": "CommittedOffset", "type": "int64", "versions": "0+",
"about": "The message offset to be committed." },
{ "name": "CommittedLeaderEpoch", "type": "int32", "versions": "6+", "default": "-1", "ignorable": true,
"about": "The leader epoch of this partition." },
// CommitTimestamp has been removed from v2 and later.
{ "name": "CommitTimestamp", "type": "int64", "versions": "1", "default": "-1",
"about": "The timestamp of the commit." },
{ "name": "CommittedMetadata", "type": "string", "versions": "0+", "nullableVersions": "0+",
"about": "Any associated metadata the client wants to keep." }
]}
]}
]
} |
Required ACL
- Read Group
Request Validation
INVALID_REQUEST is returned should the request not obey to the following invariants:
- A topic has both a name and an ID set.
Request Handling
The MemberId and the GenerationIdOrMemberEpoch are verified. FENCED_MEMBER_EPOCH or UNKNOWN_MEMBER_ID is returned accordingly.
...
Required ACL
- Describe Group
Request Validation
INVALID_REQUEST is returned should the request not obey to the following invariants:
- A topic has both a name and an ID set.
Request Handling
The MemberId and the GenerationIdOrMemberEpoch are verified. FENCED_MEMBER_EPOCH, UNKNOWN_MEMBER_ID or ILLEGAL_GENERATION is returned accordingly.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"apiKey": 9,
"type": "response",
"name": "OffsetFetchResponse",
// Version 1 is the same as version 0.
//
// Version 2 adds a top-level error code.
//
// Version 3 adds the throttle time.
//
// Starting in version 4, on quota violation, brokers send out responses before throttling.
//
// Version 5 adds the leader epoch to the committed offset.
//
// Version 6 is the first flexible version.
//
// Version 7 adds pending offset commit as new error response on partition level.
//
// Version 8 is adding support for fetching offsets for multiple groups
//
// Version 9 adds TopicId field and can return FENCED_MEMBER_EPOCH, UNKNOWN_MEMBER_ID,
// and ILLEGAL_GENERATION errors.
"validVersions": "0-8",
"flexibleVersions": "6+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "3+", "ignorable": true,
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "Topics", "type": "[]OffsetFetchResponseTopic", "versions": "0-7",
"about": "The responses per topic.", "fields": [
{ "name": "Name", "type": "string", "versions": "0-7", "entityType": "topicName",
"about": "The topic name." },
{ "name": "Partitions", "type": "[]OffsetFetchResponsePartition", "versions": "0-7",
"about": "The responses per partition", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0-7",
"about": "The partition index." },
{ "name": "CommittedOffset", "type": "int64", "versions": "0-7",
"about": "The committed message offset." },
{ "name": "CommittedLeaderEpoch", "type": "int32", "versions": "5-7", "default": "-1",
"ignorable": true, "about": "The leader epoch." },
{ "name": "Metadata", "type": "string", "versions": "0-7", "nullableVersions": "0-7",
"about": "The partition metadata." },
{ "name": "ErrorCode", "type": "int16", "versions": "0-7",
"about": "The error code, or 0 if there was no error." }
]}
]},
{ "name": "ErrorCode", "type": "int16", "versions": "2-7", "default": "0", "ignorable": true,
"about": "The top-level error code, or 0 if there was no error." },
{ "name": "Groups", "type": "[]OffsetFetchResponseGroup", "versions": "8+",
"about": "The responses per group id.", "fields": [
{ "name": "groupId", "type": "string", "versions": "8+", "entityType": "groupId",
"about": "The group ID." },
{ "name": "Topics", "type": "[]OffsetFetchResponseTopics", "versions": "8+",
"about": "The responses per topic.", "fields": [
{ "name": "Name", "type": "string", "versions": "8+", "nullableVersions": "9+", "default": "null", "entityType": "topicName",
"about": "The topic name."},
{ "name": "TopicId", "type": "uuid", "versions": "9+", "about": "The unique topic ID" },
{ "name": "Partitions", "type": "[]OffsetFetchResponsePartitions", "versions": "8+",
"about": "The responses per partition", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "8+",
"about": "The partition index." },
{ "name": "CommittedOffset", "type": "int64", "versions": "8+",
"about": "The committed message offset." },
{ "name": "CommittedLeaderEpoch", "type": "int32", "versions": "8+", "default": "-1",
"ignorable": true, "about": "The leader epoch." },
{ "name": "Metadata", "type": "string", "versions": "8+", "nullableVersions": "8+",
"about": "The partition metadata." },
{ "name": "ErrorCode", "type": "int16", "versions": "8+",
"about": "The partition-level error code, or 0 if there was no error." }
]}
]},
{ "name": "ErrorCode", "type": "int16", "versions": "8+", "default": "0",
"about": "The group-level error code, or 0 if there was no error." }
]}
]
} |
Response Handling
Upon receiving the FENCED_MEMBER_EPOCH error, the consumer retries when receiving its next heartbeat response with its member epoch.
...
- Describe Config on the group.
Request Validation
No changes.
Request Handling
The new GROUP resource type is handled.
Response Schema
No changes.
Response Handling
No changes.
AlterIncrementalConfigs API
...
- Alter Config on the group.
Request Validation
No changes.
Request Handling
The new GROUP resource type is handled.
Response Schema
No changes.
Response Handling
No changes.
Records
This section describes the new record types required for the new protocol. The storage layout is based on the data model described earlier in this document.
...