DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: Accepted
Under Discussion thread: here
Discussion Vote thread: here
JIRA: here [Change the link from KAFKA-1 to your own ticket]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
The Kafka protocol doesn't quite have sessions like many other protocols, but it does have an initial RPC used by a client when it connects to a broker which is ApiVersions. By adding optional ClusterId and NodeId information to the ApiVersions request, the receiving broker would be able to tell the client when it is attempting to make a misrouted connection.
If the client has disabled rebootstrapping by setting metadata.recovery.strategy=NONE , this checking is also disabled because the client would not be able to rebootstrap when instructed.
Public Interfaces
Kafka protocol
...
This KIP introduces version 5 which adds ClusterId and NodeId to the request.
If the client is bootstrapping, it does not supply ClusterId or NodeId. After bootstrapping, during which it learns the information from its initial Metadata response, it supplies both. When rebootstrapping begins, the client discards the cluster ID and node information that it learnt from its earlier Metadata response so that it can bootstrap from scratch.
The validation of ClusterId and NodeId is as follows:
- If neither
ClusterIdnorNodeIdis specified, the request proceeds as normal - If just one of
ClusterIdorNodeIdis specified, the request fails and the error codeINVALID_REQUESTis returned. - If both
ClusterIdandNodeIdare specified and match the receiving broker, the request proceeds as normal. - If the
ClusterIdis incorrect, the request fails and the error codeINCONSISTENT_CLUSTER_IDREBOOTSTRAP_REQUIREDis returned. This is a non-fatal error for the client which should rebootstrap. - If the
ClusterIdis correct but theNodeIdis incorrect, the request fails and the error codeREBOOTSTRAP_REQUIREDis returned. This is a non-fatal error for the client which should rebootstrap.
...
The response schema is unchanged. ApiVersions is now able to return two an additional existing errors codeserror code: REBOOTSTRAP_REQUIRED and INCONSISTENT_CLUSTER_ID .
Configuration
Common client configuration
A low importance configuration is added just in case an unforeseen situation arises where the new checks introduced by this KIP cause problems with existing deployments. Kafka clients have always supported cluster ID change without requiring restart. By clearing cluster metadata when rebootstrapping begins, Kafka clients should be able to switch cluster ID safely which could be useful in disaster recovery scenarios.
| Configuration | Description | Values |
|---|---|---|
metadata.cluster.check.enable | Whether the client should send cluster and node information when connecting to a broker to enable it to check for a misrouted connection. This configuration is ignored if rebootstrapping is disabled by setting the configuration metadata.recovery.strategy=NONE . If the client is connecting to a broker older than Apache Kafka <insert version here>, no checking is performed and this configuration has no effect. | Boolean. true (default), or false |
Compatibility, Deprecation, and Migration Plan
...
An alternative would be to introduce a new Connect RPC into the Kafka protocol, rather than extending ApiVersions. This would naturally extend to support new features such as multi-tenancy and namespaces in the future. We probably would not want to add additional context such as a namespace onto ApiVersions, so maybe we should bite the bullet now and add a new RPC. Part of the motivation of this KIP is to start the discussion in the community about the relative merits.
...