Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Initialize phase
    During a broker startup, broker calculates the initial set of orphan partitions based on the partition information from the first leaderandISR request. 
  2.  timeout/correction phase (such as 1 hour 2 hours of timeout, defined by an internal timer)
    During timeout phase, a broker updates its knowledge about partitions over time. The firstleaderandISR request the broker receives might be outdated (due to dual controllers, outdated requestsnetwork partitions, etc.). However, during the timeout phase , the broker will receive more leaderandISR requests and use partitions information from leaderandISR requests to remove partitions that the broker is responsible for from the initial orphan partition set.   We rely on the fact that, until timeout timer expires, there is at least one valid leaderandISR for any given partition hosted by the broker. 

  3. Deletion phase.
    The broker removes orphan partitions (including partition folders) whose log segments are all older than the broker default retention period.  Broker will not distinguish between the log compacted topic and time-retention topics for those partitions in orphan partition set. The default retention period of a broker is used for all orphan partitions. Broker only removes orphan partitions whose log segments are all older than the default retention period. This is to ensure broker will not try to delete new data. If some orphan partitions cannot be removed immediately because the retention period has not been reached, a new deletion will be scheduled again to perform deletion.

...