THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Current state: "Draft" [One of "Under Discussion", "Accepted", "Rejected"]
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
All the request and the response types have been analysis to identify the potential gaps in the API. Especially, we have looked at all the "bytes" fields in the API.
| Type | Field (type) | Observation | Status |
|---|---|---|---|
| DelegationToken API | |||
| CreateDelegationTokenResponse |
| The `Hmac` field contains the HMAC of the delegation token. It is computed by the broker (HmacSHA512) when the delegation token is created, returned in the `CreateDelegationTokenResponse` and used in all the other requests/responses. There is not point in parsing it in a proxy. | OK |
| DescribeDelegationTokenResponse |
| ||
| ExpireDelegationTokenRequest |
| ||
| RenewDelegationTokenRequest |
| ||
| SaslAuthenticate API | |||
| SaslAuthenticateRequest |
| The `AuthBytes` field contains the SASL authentication bytes, as defined by the SASL protocol. It could be parsed independently by following the SASL protocol. | OK |
| SaslAuthenticateResponse |
| ||
| Produce/Fetch APIs | |||
| ProduceRequest |
| The `Records` field contains the so called Records Batches. They can be parsed using the Records Batched format as defined here. | OK |
| FetchResponse |
| ||
| Group API | |||
| JoinGroupRequest |
| The `Metadata` fields contain the protocol metadata of the member who would like to join the group. The request contains a `ProtocolType` field which indicate the protocol used to populate the `Metadata` fields. The group API has been designed to allow any protocol types. In practice, there are few well known types such as "consumer" or "connect" which are respectively used by Kafka Consumer and Kafka Connect. The request could be handled independently if the protocol type is known by the proxy. Note that there are cases where the broker also parses the metadata (e.g. KIP-496: Administrative API to delete consumer offsets). | OK |
| JoinGroupResponse |
| The `Metadata` field contain the protocol metadata of a member of the group. Metadata of all the members are sent back to the group leader to let him compute the assignments for the group. The response does not contain any indication of the protocol type used. It was not put in the response because the members already have the information on their end. It means that the response could not be handled independently without prior knowledge captured from the JoinGroupRequest. | GAP |
| SyncGroupRequest |
| The `Assignment` field contain the assignment for a member of the group. The group leader computes all the assignments foreach member and send them to the group coordinator. The request does not contain any indication of the protocol type used because both ends have already the information. It means that the request could not be handled independently without knowing the prior information exchanged between the members and the coordinator. | GAP |
| SyncGroupResponse |
| The `Assignment` field contain the assignment for a member of the group. The group coordinator sends back the information to each member. | GAP |
| DescribeGroupsResponse |
| The `MemberMetadata` and `MemberAssignment` contain the metadata and the assignment of a member. The response contains a `ProtocolType` field which indicate the protocol used by the group. The response can be handled independently. | OK |
Proposed Changes
In order to fill in the gaps which have identified in the Group API, we propose to explicitly put the `ProtocolType` in all the requests and responses which contain the `Metadata` and/or the `Assignment`. Namely, we propose to add it in the `JoinGroupResponse`, the `SyncGroupRequest` and the `SyncGroupResponse`. See Public Interfaces for details.
...