Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This KIP unifies these two checkpoints by moving the starting metadata from bootstrap.checkpoint to the zero checkpoint. The main advantage of using the zero checkpoint is that it integrates with the rest of the checkpoint mechanisms like checkpoint loading (RaftClient.Listener#handleLoadSnapshot) and checkpoint deletion introduced in KIP-630: Kafka Raft Snapshot. For example, not deleting the bootstrap.checkpoint has cause issues with Kafka startup logic as documented in

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-19191
.

Currently, these two snapshots checkpoints that handle bootstrapping metadata can be viewed as "logically separate," and this KIP seeks to unify them under the zero checkpoint in KRaft.

...

When the controller handles RaftClient.Listener#handleLoadSnapshot if the snapshot checkpoint id has an epoch of 0 and a base offset of 0, the controller will consider these records as the bootstrapping records. The controller will rewrite bootstrap records to the log if they haven't been successfully written in the past. If the controller doesn't load a snapshot checkpoint at epoch 0 and offset 0, the controller will load the bootstrap.checkpoint and rewrite the bootstrapping record to the cluster metadata partition if they haven't been successfully written in the past.

...

  1. bootstrap.checkpoint exist with metadata records and the zero checkpoint doesn't exists - In this case the controller will behave as it does today. The controller will be able to identify this case because RaftClient.Listener#handleLoadSnapshot won't ask to load the zero snapshotcheckpoint.
  2. bootstrap.checkpoint exist with metadata records and the zero checkpoint exists but doesn't contain any metadata records - In this case the controller will behave as it does today. The controller will be able to identify this case because RaftClient.Listener#handleLoadSnapshot will ask to load the zero checkpoint but it will be empty, no metadata records.
  3. bootstrap.checkpoint doesn't exist and the zero checkpoint exists with metadata records - In this case the controller will use the zero checkpoint's metadata records to and write them to the log in a transaction or single atomic batch like the controller does pre-KIP-1170.
  4. bootstrap.checkpoint exist and the zero checkpoint exists with metadata records - This should not be possible from a formatting point of view but the active controller will handle this case the same as bullet 3 but with the addition of writing a WARN message to the controller log.

...