...

• The command might issue warning under certain situations, such as
  • "WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected"
  • "WARNING: This feature is only enabled with Metadata version above 10 and delete.topic.partition.enable turned on"
  • "WARNING: The topic is currently under changes"

Proposed Changes

We propose to add new status for a partition that indicates if a partition is to be removed. Specifically at any time a partition can be 

• online (code=0): it means the partition is normal

• offline (code=1): it means the partition should be excluded by SDK client for both message production and consumption. When the data is purged this partition will be removed from the topics metadata.

• readOnly (code=-1): it means the partition should be filtered and not written to, but consumption is not impacted. After the retention period has passed the status will transit to offline.

The status is stored in zookeeper:

{"controller_epoch":1,"leader":0,"version":1,"leader_epoch":0,"isr":[0], "status_code": 0}

The following change in Controller will be made:

• In zookeeper a path /admin/delete_topic_partitions will be added. The format of the node is ()

• In the controller monitoring path, register DeleteTopicPartitionHandler to monitor the zookeeper event; add a controller event Added controller event TopicPartitionDeletion
• Add a class TopicPartitionDeletionManager TopicPartitionDeleteManager to handle TopicPartitionDeletion event
• When KafkaController starts, a scheduleDelayDeletePartitionTask is scheduled periodically to check retention for delayed deletion.

The workflow involving TopicPartitionDeletionManager class is summarized as below:

...