...
The following table shows the list of methods that will need to change in order to support the max.broker.partitions and max.partitions configurations. (We skip a few internal methods for the sake of simplicity.)
Method name | Description of what the method does currently | Context in which used | Relevant methods which directly depend on this one | Relevant methods on which this one is directly dependent | Description of what the method does currently | Context in which used |
---|---|---|---|---|---|---|
`AdminUtils.assignReplicasToBrokers``AdminZkClient.createTopic` `AdminZkClient.addPartitions` `AdminManager.createTopics` `ReassignPartitionsCommand.generateAssignment` |
|
| `AdminZkClient. createTopicWithAssignment`createTopic` `AdminZkClient.createTopic`addPartitions` `AdminManager.createTopics``ZookeeperTopicService.createTopic` `ReassignPartitionsCommand.generateAssignment` | |||
`AdminZkClient.createTopicWithAssignment` | Creates the ZooKeeper znodes required for topic-specific configuration and replica assignments for the partitions of the topic. |
| `AdminZkClient.createTopic``KafkaApis `AdminManager.createTopic`createTopics` `ZookeeperTopicService.createTopic``AdminUtils | |||
`AdminZkClient.assignReplicasToBrokers`createTopic` | `AdminZkClient.createTopicWithAssignment` Computes replica assignment using `AdminUtils.assignReplicasToBrokers` and then reuses `AdminZkClient.createTopicWithAssignment`. |
| `AdminZkClient.addPartitions` | `AdminManager.createPartitions``KafkaApis.createTopic` `ZookeeperTopicService.alterTopic`createTopic` | `AdminUtils.assignReplicasToBrokers` `AdminZkClient.createTopicWithAssignment` | |
`AdminZkClient.addPartitions` |
|
| `AdminManager. createTopics`createPartitions` `KafkaApis`ZookeeperTopicService. handleCreateTopicsRequest`alterTopic` | `AdminUtils.assignReplicasToBrokers` | ||
`AdminZkClient`AdminManager.createTopicWithAssignment`createTopics` |
|
| `AdminManager`KafkaApis.createPartitions`handleCreateTopicsRequest``KafkaApis | `AdminUtils. handleCreatePartitionsRequest`assignReplicasToBrokers` `AdminZkClient. addPartitions`createTopicWithAssignment` | ||
`AdminManager.createPartitions` | Used exclusively by `KafkaApis.handleCreatePartitionsRequest` to create partitions on an existing topic. |
| `KafkaController`KafkaApis.onPartitionReassignment`handleCreatePartitionsRequest``KafkaApis | `AdminZkClient.handleAlterPartitionReassignmentsRequest`addPartitions` | ||
`KafkaController.onPartitionReassignment`(not quite directly, but the stack trace in the middle is not relevant) | Handles all the modifications required on ZooKeeper znodes and sending API requests required for moving partitions from some brokers to others.APIto others. |
| `KafkaApis.handleAlterPartitionReassignmentsRequest` (not quite directly, but the stack trace in the middle is not relevant) | |||
`KafkaApis.handleCreateTopicsRequest``AdminManager.createTopics` | Handles the CreateTopics API request sent to a broker, if that broker is the controller. |
| `AdminManager.createTopics` | |||
`KafkaApis.handleCreatePartitionsRequest``AdminManager.createPartitions` | Handles the CreatePartitions API request sent to a broker, if that broker is the controller. |
| `AdminManager.createPartitions` | |||
`KafkaApis.handleAlterPartitionReassignmentsRequest``KafkaController.onPartitionReassignment` (not quite directly, but the stack trace in the middle is not relevant) | Handles the AlterPartitionReassignments API request sent to a broker, if that broker is the controller. |
| `KafkaApis.createTopic` | `KafkaController.onPartitionReassignment` `KafkaApis.handleTopicMetadataRequest` `KafkaApis.handleFindCoordinatorRequest` (not quite directly, but the stack trace in the middle is not relevant) `AdminZkClient | ||
`KafkaApis.createTopic` |
|
| `KafkaApis.handleTopicMetadataRequest` `KafkaApis.createTopic`handleFindCoordinatorRequest` (not quite directly, but the stack trace in the middle is not relevant) | `AdminZkClient.createTopic` | ||
`KafkaApis.handleTopicMetadataRequest` | Handles the Metadata API request sent to a broker. |
| `KafkaApis.createTopic` (not quite directly, but the stack trace in the middle is not relevant) | |||
`KafkaApis.handleFindCoordinatorRequest` | Handles the FindCoordinator API request sent to a broker. |
| `ZookeeperTopicService `KafkaApis.createTopic` (not quite directly, but the stack trace in the middle is not relevant) | |||
`ZookeeperTopicService.createTopic` `AdminZkClient.createTopic` `AdminZkClient.createTopicWithAssignment` |
|
| `ZookeeperTopicService `AdminZkClient. alterTopic`createTopic` `AdminZkClient. addPartitions`createTopicWithAssignment` | |||
`ZookeeperTopicService.alterTopic` |
|
| `AdminZkClient.addPartitions` | |||
`ReassignPartitionsCommand.generateAssignment` | `AdminUtils.assignReplicasToBrokers` | Used by the ./kafka-reassign-partitions.sh admin tool to generate a replica assignment of partitions for the specified topics onto the set of specified brokers. |
| `AdminUtils.assignReplicasToBrokers` |
For all the methods in the above table that are used in the context of both Kafka API request handling paths and ZooKeeper-based admin tools (`AdminUtils.assignReplicasToBrokers`, `AdminZkClient.createTopicWithAssignment`, `AdminZkClient.createTopic` and `AdminZkClient.addPartitions`), we will pass the values for maximum number of partitions per broker, maximum number of partitions overall, and the current number of partitions for each broker as arguments.
...