Page History

...

The controller receives a delete request for a topic. It goes through the existing protocol of deletion and it makes all the replicas offline to stop taking any fetch requests.  After all the replicas reach the offline state, the controller publishes an event to the remote log metadata topic by marking the topic as deleted.  With KIP-516, topics are represented with uuid, and topics can be deleted asynchronously. This allows the remote logs can be garbage collected later by publishing the deletion marker into the remote log metadata topic.

Image Modified

RemotePartitionRemover instance is created on the leader for each of the remote log segment metadata topic partitions. It consumes messages from that partitions and filters the delete partition events which need to be processed. It also maintains a committed offset for this instance to handle leader failovers to a different replica so that it can start processing the messages where it left off. 

...

1. The controller publishes delete_partition_marked event to say that the partition is marked for deletion. There can be multiple events published when the controller restarts or failover and this event will be deduplicated by RLC. 
2. RPM receives the delete_partition_marked and processes it if it is not yet processed earlier.
3. RPM publishes an event delete_partition_started that indicates the partition deletion has already been started. 
4. RPMgets all the remote log segments for the partition and each of these remote log segments are deleted with the next steps.
5. Publish delete_segment_started event with the segment id. 
6. RPM deletes the segment using RSM 
7. Publish delete_segment_finished event with segment id once it is successful. 
8. Publish delete_partition_finished once all the segments have been deleted successfully.

Image RemovedImage Added

Recovering  from remote log metadata topic partitions truncation

...