THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
For each type of Admin Request a separate type of Wire protocol message is created.
Currently there are It is proposed to add / modify these 3 types of requests:
- Topic commands which include
CreateTopic(Request | Response),AlterTopic,DeleteTopic,DescribeTopic,ListTopics. - Replication tools -
ReassingPartition,VerifyReassingPartitions;PreferredReplicaLeaderElection - A special type of request to support Admin commands -
ClusterMetadata [1]enrichedTopicMetadataRequest(to addcontrollerId)
Please find details under specific RQ/RP schema proposal.
...
The same notation as in A Guide To The Kafka Protocol is used here.
All admin messages listed below are required to be sent only to Controller broker. Only controller will process such messages. If Admin message is sent to an ordinary broker a special error code is returned (code 22). In case of other failure during processing message AdminRequestFailedError is returned [2].
Error | Code | Description |
|---|---|---|
| 21 | Unexpected error occurred while processing Admin request. |
NotControllerForAdminRequest | 22 | Target broker (id=<this_broker_id>) is not serving a controller's role. |
ClusterMetadata Schema
Cluster Metadata Request
ClusterMetadataRequest => |
Cluster Metadata Response
ClusterMetadataResponse => ErrorCode [Broker] ?(Controller)Broker => NodeId Host PortNodeId => int32 Port => int32 Controller => Broker |
ClusteMetadataRequest is a request with no arguments.
ClusterMetadataResponse holds error code (0 in case of successful result), list of brokers in cluster and optionally broker serving a Controller's role (returning empty Controller most likely means either error during request processing or cluster being in some intermediate state).
ClusterMetadataRequest is required for admin clients to get the Kafka brokers, specifically the controller's location, as only controller may execute admin command.
Topic Admin Schema
Protocol Errors
It is proposed to add these error codes to the protocol.
Error | Code | Description | Requests |
|---|---|---|---|
NotControllerReceivedAdminRequest | 1001 | Target broker is not serving a controller's role. | For all Admin requests |
TopicAlreadyExists | 1002 | Topic with this name already exists. | CreateTopicRequest |
InvalidArgumentPartitions | 1003 | Either partition field is invalid (e.g. negative), or not defined when needed. | CreateTopicRequest, AlterTopicRequest |
DecreasePartitionsNotAllowed | 1004 | Invalid partitions argument: decrease partitions is prohibited. | AlterTopicRequest |
InvalidArgumentReplicationFactor | 1005 | Either replication-factor field is invalid (e.g. negative), or not defined when needed. | CreateTopicRequest |
InvalidArgumentReplicaAssignment | 1006 | Either replication-factor field is invalid (e.g. contains duplicates), or not defined when needed. |
|
InvalidTopicConfig | 1007 | Either topic-level config setting or value is incorrect. | CreateTopicRequest, AlterTopicRequest |
PreferredReplicaLeaderElectionInProgress | 1008 | Preferred replica leader election procedure has been already started. | PreferredReplicaLeaderElectionRequest |
InvalidArgumentPreferredReplicaElectionData | 1009 | Preferred replica leader election data is in invalid (bad json, duplicates etc). | PreferredReplicaLeaderElectionRequest |
ReassignPartitionsInProgress | 1010 | Reassign partitions procedure has been already started. | ReassignPartitionsRequest |
Generally, the Admin Client (see section 3) or another request dispatcher should have enough context to provide descriptive error message.
E.g. in case of receiving InvalidArgumentPartitions client will be able to define:
a) upon AlterTopicRequest: this happened because user provided incorrect partitions argument (e.g. negative)
b) upon CreateTopicRequest: this happened because user provided replication-factor but not provided partitions argument
ClusterMetadata Schema
Cluster Metadata Request
ClusterMetadataRequest => |
Cluster Metadata Response
ClusterMetadataResponse => ErrorCode [Broker] ?(Controller)Broker => NodeId Host PortNodeId => int32 Port => int32 Controller => Broker |
ClusteMetadataRequest is a request with no arguments.
ClusterMetadataResponse holds error code (0 in case of successful result), list of brokers in cluster and optionally broker serving a Controller's role (returning empty Controller most likely means either error during request processing or cluster being in some intermediate state).
ClusterMetadataRequest is required for admin clients to get the Kafka brokers, specifically the controller's location, as only controller may execute admin command.
Topic Admin Schema
The idea is to introduce Wire protocol messages that cover all topic commands (create, alter, delete, list, describe). The motivation behind the proposed schema is the following:
1) Topic commands must inherit options from TopicCommand tool
2) If some of the options are not used in particular command (e.g. ReplicaAssignment in CreateTopicRequest) - the special marker value is used instead (e.g. in case of ReplicaAssignment - empty string)
3) Topic commands must support batching and provide command execution result per-topic
4) Topic commands can be executed only on a broker serving a controller's role - in case request is sent to an ordinary broker - a request-level error should reflect that
Create Topic Request
CreateTopicRequest => [TopicName ?( Partitions ) ?( Replicas ) ?( ReplicaAssignment ) [ConfigEntry]]TopicName => stringPartitions => int32Replicas => int32ReplicaAssignment => stringConfigKey => stringConfigValue => string |
...