THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Current state: Under Discussion
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here [Change the link from KAFKA-1 to your own ticket]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
This KIP is the broker-side complement of KIP-511.
Public Interfaces
...
DescribeCluster API
Bump ApiVersionsRequest DescribeClusterRequest and ApiVersionsResponse DescribeClusterResponse to v4 v1 and add two new required fields, BrokerSoftwareName and BrokerSoftwareVersion, to the response.
| Code Block | ||
|---|---|---|
| ||
{
"apiKey": 1860,
"type": "response",
"name": "ApiVersionsResponseDescribeClusterResponse",
// ...
"validVersions": "0-41", // BUMPED TO VERSION 1
"flexibleVersions": "30+",
"fields": [
{ "name": "ErrorCodeThrottleTimeMs", "type": "int16int32", "versions": "0+",
"about": "The top-levelduration errorin code." },
{ "name": "ApiKeys", "type": "[]ApiVersion", "versions": "0+",
"about": "The APIs supported by the broker.", "fields": [milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ApiKeyErrorCode", "type": "int16", "versions": "0+", "mapKey": true,
"about": "The API index. top-level error code, or 0 if there was no error" },
{ "name": "MinVersionErrorMessage", "type": "int16string", "versions": "0+", "nullableVersions": "0+",
"default": "null",
"about": "The minimumtop-level supportederror version, inclusivemessage, or null if there was no error." },
{ "name": "MaxVersionClusterId", "type": "int16string", "versions": "0+",
"about": "The maximumcluster supportedID version, inclusive." }
]that responding broker belongs to." },
{ "name": "ThrottleTimeMsControllerId", "type": "int32", "versions": "0+", "default": "-1+", "ignorableentityType": true"brokerId",
"about": "The durationID in milliseconds for which of the request was throttled due to a quota violation, or zero if the request did not violate any quotacontroller broker." },
{ "name": "SupportedFeaturesBrokers", "type": "[]SupportedFeatureKey", "ignorable": true,
DescribeClusterBroker", "versions": "3+", "tag": 0, "taggedVersions": "3+",
"about": "FeaturesEach supportedbroker byin the brokerresponse.",
"fields": [
{ "name": "NameBrokerId", "type": "stringint32", "versions": "30+", "mapKey": true,
"entityType": "brokerId",
"about": "The name of the featurebroker ID." },
{ "name": "MinVersionHost", "type": "int16string", "versions": "30+",
"about": "The minimum supported version for the featurebroker hostname." },
{ "name": "MaxVersionPort", "type": "int16int32", "versions": "30+",
"about": "The maximum supported version for the featurebroker port." },
]
},
{ "name": "FinalizedFeaturesEpochRack", "type": "int64string", "versions": "30+",
"tagnullableVersions": 1, "taggedVersions": "3"0+", "default": "-1null",
"ignorable": true,
"about": "The monotonicallyrack increasingof epoch for the finalizedbroker, featuresor information.null Validif valuesit arehas >=not 0.been Aassigned valueto of -1 is special and represents unknown epocha rack." },
{ "name": "FinalizedFeatures", "type": "[]FinalizedFeatureKey", "ignorable": true]},
// -------------- START NEW "versions": "3+", "tag": 2, "taggedVersions": "3+",FIELD --------------
{ "aboutname": "List of cluster-wide finalized features. The information is valid only if FinalizedFeaturesEpoch >= 0.",
"fields": [
{"name": "Name", "type": "string", "versions": "3BrokerSoftwareName", "type": "string", "versions": "1+", "nullableVersions": "1+", "mapKeyignorable": true,
"about": "The name of the feature broker replying to this request." },
{ {"name": "MaxVersionLevelBrokerSoftwareVersion", "type": "int16string", "versions": "31+",
"aboutnullableVersions": "The cluster-wide finalized max version level for the feature."},
{"name": "MinVersionLevel", "type": "int16", "versions": "3+"1+", "ignorable": true,
"about": "The cluster-wideversion finalizedof minthe versionbroker levelreplying forto thethis featurerequest."}
]
},
// -------------- STARTEND NEW FIELD --------------
{ "name": "BrokerSoftwareNameClusterAuthorizedOperations", "type": "nullable-stringint32", "versions": "40+", "ignorabledefault": true"-2147483648",
"about": "The name of the broker32-bit bitfield to represent authorized operations for this cluster." },
]
} |
AdminClient API
The DescribeClusterResponse class will be updated to add two methods:
| Code Block | ||
|---|---|---|
| ||
public class { "name": "BrokerSoftwareVersion", "type": "nullable-string", "versions": "4+", "ignorable": true, "about": "The version of the broker." } // -------------- END NEW FIELD -------------- ]DescribeClusterResponse extends AbstractResponse { // ... public String brokerSoftwareName() { return data.brokerSoftwareName(); } public String brokerSoftwareVersion() { return data.brokerSoftwareVersion(); } } |
Proposed Changes
This KIP proposes two new required fields in the ApiVersions DescribeCluster response to permanently remove the need to "guess" a version going forwardbased on the ApiVersions response, and to add the ability to determine the name of the software the client is talking to. For Kafka, the idea is to return "Kafka" for the BrokerSoftwareName, and CURRENT_BROKER_VERSION for the BrokerSoftwareVersion. For other Kafka-like replacements, they can return something like "Kinesis" and ${kinesis_version}, or "Redpanda" and ${redpanda_version}.
...
Put BrokerSoftwareName and BrokerSoftwareVersion in the ResponseHeader
This One alternative is mentioned to parallel KIP-511, but there is even less reason to include the broker name and version in every response. Clients are not interested in these pieces of information on every response, so we should not require it in every response. The ApiVersions DescribeCluster response is the perfect place to put these fields: the client asks for ApiVersions likely once only interested clients will ask for the cluster metadata. Clients never need to know the broker name nor version otherwise.
Put BrokerSoftwareName and BrokerSoftwareVersion in the ApiVersions response
This KIP originally proposed placing these two new fields in ApiVersions. After discussion, we want to avoid the risk of the BrokerSoftwareVersion being misused by clients which opt into or out of behavior. The easiest way to avoid misuse is to simply not make misuse possible. As well, the average producer & consumer is not interested in the broker name nor version. We should only return these fields in a less-frequently-used, more admin request.
Return the name and version per broker in the DescribeCluster response
The broker software name and broker software version are not available centrally. These are going to be per-broker fields. A single broker cannot return the software name and version for all other brokers; if a client is interested in the name and version for all brokers, the client must request all brokersper connection, and in this single request, we can determine the text representation of the broker we are talking to.
Put BrokerSoftwareName and BrokerSoftwareVersion in the ResponseHeader but send it only once
Again paralleling KIP-511, these fields could be sent only in the first ResponseHeader to save bytes in subsequent requests. Lik KIP-511, it would be weird to have the information in random requests and it would make clients inconsistent.
...