DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: DraftUnder discussion
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here [Change the link from KAFKA-1 to your own ticket]
...
Although a topic name limit is applied by the broker, it is still possible to get an ugly client-side exception . As a result, it seems appropriate to apply the proper limit on the broker, and apply a looser sanity check on the clienteven in the case of topic names. For other identifiers such as group IDs, it is only the serialization limit which prevents excessively long identifiers because there is no broker-side limit defined. This KIP introduces broker-side limits for all identifiers.
Proposed Changes
This KIP proposes enforcing maximum lengths for all resource names and identifiers, starting in Apache Kafka 5.0. The suggested maximum values are chosen to be sufficiently high that they are very unlikely to impact any existing users, while providing stronger enforcement of limits in the Kafka protocol.
...
The following table summarises the proposal:
Resource name or identifier | Current limit |
|---|
Proposed broker limit | Proposed error code when broker limit breached | Notes | |
|---|---|---|---|
Topic name | 249 characters |
32767 characters
249 characters -no change |
| |
Group ID | Serialization limit of a string |
249 characters |
| ||
Group member ID | Serialization limit of a string |
36 characters |
| The group member ID is not specified by the user. In KIPs 848, 932 and 1071, the client library is intended to generate a UUID to be used as the member ID. As a result, the maximum should be the length of a serialized UUID, such as | |
Group instance ID | Serialization limit of a string |
249 characters |
| ||
Client ID | Serialization limit of a string |
32767 characters
249 characters |
| |
Transactional ID | Serialization limit of a string |
249 characters |
| ||
Offset commit metadata | 4096 bytes |
32767 characters
4096 bytes - no change |
| Controlled by the |
Rack ID | Serialization limit of a string |
249 characters |
| ||
Resource name for configs and ACLs | Serialization limit of a string |
249 characters |
| ||
ACL principal | Serialization limit of a string |
32767 characters
249 characters |
| |
ACL host | Serialization limit of a string |
32767 characters
249 characters |
|
A single new error code RESOURCE_IDENTIFIER_TOO_LARGE is defined to use whenever any of the limits is breached, as opposed to a separate error code for each case. The exception message will provide more detail about which identifier was too long.
...
Configuration
Broker configuration
| Configuration | Description | Values |
|---|---|---|
resource.identifier.limit.enable | Whether the cluster enforces maximum sizes for resource names and identifiers. | In Kafka 4.x: default In Kafka 5.0 and later: default |
Kafka protocol changes
Error codes
...
This new error code can be returned in the response from any Kafka protocol request which breaches the limit for a resource name or identifier. The versions of the requests and responses are not bumped by this KIPAs a result, the RPC request and response versions for all affected RPCs must be bumped since this ensures that the client receiving the new error code will know how to interpret it. The RPCs affected will be any whose request schema includes a string-based identifier from the table above.
The associated exception ResourceIdentifierTooLargeException is a subclass of ApiException . It is not a retriable exception.
The exception will be thrown in the client if the supplied value exceeds the serialization sanity check (32767 characters)For clients which do not support RPC versions which can return the new RESOURCE_IDENTIFIER_TOO_LARGE error code, the error code will be INVALID_REQUEST . The idea is that the client-side changes for this KIP are made long in advance of Apache Kafka 5.0, with the result that most clients will support the new error code by the time production clusters are enforcing the limit.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?
Test Plan
Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
Rejected Alternatives
The idea is that this KIP does not have any user impact at all, in the belief that users are already using identifiers within the new limits.
In Apache Kafka 5.0, the new limits will be applied, unless the resource.identifier.limit.enable config is set to "false" .
Test Plan
The code will be tested using a combination of unit tests, integration tests and system tests.
Rejected Alternatives
None consideredIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.