Page History

...

Currently, the first LeaderAndIsrRequest sent to a broker by a controller contains all TopicPartitions that a broker is a replica for. We will formalize this behavior by also including an type enum to denote the type of LeaderAndIsrRequest.

When type = FULL, the broker can reconcile its local state on disk with the request, and safely stage deletions for any partitions that are present on disk and are not contained in the request. This may include cases where a TopicPartition is not present in the LeaderAndIsrRequest, or it may be due to a topic partition containing a topic ID that does not match the local topic partition stored on the broker. Such reconciliation may also be necessary if type = INCREMENTAL and the topic ID set on a local partition does not match the topic ID contained in the request. When type = UNSPECIFIED, the request will be treated in a way that allows for backwards compatibility with older request types.

Deletions of stale partitions triggered by LeaderAndIsrRequest(s) will:

1. Log a warningpartitions that will be deleted and the time they will be deleted at WARN level.
2. Move the partition's directory to log.dir/deleting/{topic_id}_{partition}
3. A final deletion event will be scheduled for delete.stale.topic.delay.ms ms after the LeaderAndIsrRequest was first received. This will clear the deleting directory of the partition's contents

...

The following configuration options will be added:

Storage

Partition Metadata file

...