Page History

...

The min commit offset is the minimal commit offset of one partition of all the consumer groups. we can simply use the simple consumer’s API to find every consumer group’s commit offset, which had subscribed the topic containing the to-be deleted log segment. To  We can handle old consumer and new consumer in one function to query the commit offset like the consumer-offset-checker tool, and can handle potential failures like leader change or coordinator down. To avoid deleting the log segment which are not really consumed, whenever encounter commit offset querying exception, we will set the min commit offset to -1.

When the min commit offset is -1, we will skip the consumed log deletion procedure.

Filter Log Segment

After a valid min commit offset is found, we begin to filter the log segments which are out of date and have been consumed by all consumer groups.

...