...
Public Interfaces
- Changes to Wire Protocol:
...
Extending protocol with these messages:
- CreateTopic(Request | Response), AlterTopic, DeleteTopic, ListTopics, DescribeTopic
- PreferredLeaderReplicaElection
- ReassignPartitions, VerifyReassignPartitions
- New client: AdminClient - a Wire Protocol client for administrative operations
- New CLI tool: Kafka Shell - a single unified command line interface for administrative operations
...
New Protocol Errors
It is proposed to add these error codes to the protocol.
Error | Code | Description | Requests |
---|---|---|---|
TopicAlreadyExists | 1001 | Topic with this name already exists. | CreateTopicRequest |
InvalidArgumentPartitions | 1002 | Either partition field is invalid (e.g. negative), or not defined when needed. | CreateTopicRequest , AlterTopicRequest |
DecreasePartitionsNotAllowed | 1003 | Invalid partitions argument: decreasing partitions is prohibited. | AlterTopicRequest |
InvalidArgumentReplicationFactor | 1004 | Either replication-factor field is invalid (e.g. negative), or not defined when needed. | CreateTopicRequest |
InvalidArgumentReplicaAssignment | 1005 | Either replication-factor field is invalid (e.g. contains duplicates), or not defined when needed. |
|
InvalidTopicConfig | 1006 | Either topic-level config setting or value is incorrect. | CreateTopicRequest , AlterTopicRequest |
PreferredReplicaLeaderElectionInProgress | 1007 | Preferred replica leader election procedure has been already started. | PreferredReplicaLeaderElectionRequest |
InvalidArgumentPreferredReplicaElectionData | 1008 | Preferred replica leader election data is in invalid (bad json, duplicates etc). | PreferredReplicaLeaderElectionRequest |
ReassignPartitionsInProgress | 1009 | Reassign partitions procedure has been already started. | ReassignPartitionsRequest |
Generally, the Admin Client (see section 3) or another request dispatcher should have enough context to provide descriptive error message.
Topic Admin Schema
The idea is to introduce Wire protocol messages that cover all topic commands (create, alter, delete, list, describe). The motivation behind the proposed schema is the following:
1) Topic commands must inherit options from TopicCommand tool
2) If some of the options are not used in particular command (e.g. ReplicaAssignment
in CreateTopicRequest
) - the special marker value is used instead (e.g. in case of ReplicaAssignment
- empty string)
3) Topic commands must support batching and provide command execution result per-topic
4) Topic commands are asynchronous - the request to create/alter/delete just initiates the corresponding commands and returns immediately
5) Topic commands can be executed only on a broker serving a controller's role - in case request is sent to an ordinary broker - a request-level error should reflect that
Create Topic Request
CreateTopicRequest => [TopicName Partitions Replicas ReplicaAssignment [ConfigEntry]] TopicName => string Partitions => int32 Replicas => int32 ReplicaAssignment => [Topic [PartitionId [ReplicaId]]]
ConfigKey => string ConfigValue => string |
CreateTopicRequest
requires topic name and either (Partitions
+Replicas
) or ReplicasAssignment
to create a topic. A special value -1
should be used to denote an empty value for Partitions
and Replicas
. Also user will be able to specify topic-level configs to create for the created topic with (to use default an empty array should be provided).The (Partitions, Replicas)/ReplicaAssignment
semantics is the following:
ReplicaAssignment
is specified - Partitions
and Replicas
are not taken into account, topic is created with provided replica assignment and number of topics and replication factor are defined from ReplicaAssignment
ReplicaAssignment
is empty - number of topic partitions and replication factor must be defined with Partitions
and Replicas
, the replica assignment for topic is automatically generated on server...