THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- Topic commands which include
CreateTopic(Request | Response),AlterTopic,DeleteTopic,DescribeTopic,ListTopics. - Replication tools -
ReassingPartition,VerifyReassingPartitions;PreferredReplicaLeaderElection - A special type of request to support Admin commands -
ClusterMetadataClusterMetadata [1]
Please find details under specific RQ/RP schema proposal.
...
The same notation as in A Guide To The Kafka Protocol is used here. The only difference - new Kafka Protocol metatype - MaybeOf ("?" in notation), when used means value is optional in message. To define value existence special control byte is prepended before each value (0 - field is absent, otherwise - read value normally) [12].
All admin messages listed below are required to be sent only to Controller broker. Only controller will process such messages. If Admin message is sent to an ordinary broker a special error code is returned (code 22). In case of other failure during processing message AdminRequestFailedError is returned [23].
Error | Code | Description |
|---|---|---|
| 21 | Unexpected error occurred while processing Admin request. |
NotControllerForAdminRequest | 22 | Target broker (id=<this_broker_id>) is not serving a controller's role. |
ClusterMetadata
Schema [3]Schema
Cluster Metadata Request
ClusterMetadataRequest => |
Cluster Metadata Response
ClusterMetadataResponse => ErrorCode [Broker] ?(Controller)Broker => NodeId Host PortNodeId => int32 Port => int32 Controller => Broker |
ClusteMetadataRequest is a request with no arguments.
ClusterMetadataResponse holds error code (0 in case of successful result), list of brokers in cluster and optionally broker serving a Controller's role (returning empty Controller most likely means either error during request processing or cluster being in some intermediate state).
ClusterMetadataRequest is required for admin clients to get the Kafka brokers, specifically the controller's location, as only controller may execute admin command [2].
Topic Admin Schema
...
# Start kafka.sh in shell modebin/kafka.sh --shell --broker-list <host1 : port1>Connected to Kafka Controller at <host2 : port2>.kafka> create-topic --topic my_topic --partitions 5 --replication-factor 3Topic "my_topic" is created.kafka> alter-topic --topic my_topic --partitions 10Topic "my_topic" is changed.# Switch topic contextkafka> topic my_topicSwitched to "my_topic" topic.# Execute topic command for switched topickafka my_topic> describe-topic"my-topic" details:Topic: my_topic Partitions: 10 ...# Switch off topic contextkafka my_topic> topickafka> |
Open questions:
ClusterMetadataduplicatesTopicMetadata- we can extendTopicMetadatawith controllerId information and probably something else. Other alternative - is a generic server-side re-routing facility (see KAFKA-1912 for details).- People expressed concerns about optional
MaybeOftype because it is inconsistent with the way we currently handle empty values in the Wire Protocol. - We might extend error codes to fulfill all possible failures and give up using
outcome/errorDescriptionfield as a generic result description.ClusterMetadataduplicatesTopicMetadata- we can extendTopicMetadatawith controllerId information and probably something else. Other alternative - is a generic server-side re-routing facility (see KAFKA-1912 for details). - It is proposed to create a separate ticket to rework topic command to execute commands directly by the controller instead of using zookeeper admin path to notify controller about the change.
AdminClientmay need to support batching admin operations. It is considered whether we can cover it with allowing user to supply a regexp for topic name inAlterTopic,DeleteTopic,DescribeTopicrequests (similarly toTopicCommand.scala)
...