This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Current state: Under Discussion
Discussion thread: TBD
JIRA: TBD
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
KIP-853: KRaft Controller Membership Changes added support for bootstrapping the KRaft state while added support for bootstrapping the metadata state. In other words, the zero checkpoint (00000000000000000000-0000000000.checkpoint) contains the starting state for KRaft while bootstrap.checkpoint contains the starting state for the cluster metadata.
This KIP unifies these two checkpoints by moving the starting metadata from bootstrap.checkpoint to the zero checkpoint. The main advantage of using the zero checkpoint is that it integrates with the rest of the checkpoint mechanisms like checkpoint loading (RaftClient.Listener#handleLoadSnapshot) and checkpoint deletion introduced in KIP-630: Kafka Raft Snapshot. For example, not deleting the bootstrap.checkpoint has cause issues with Kafka startup logic as documented in .
Currently, these two snapshots that handle bootstrapping metadata can be viewed as "logically separate," and this KIP seeks to unify them under the zero checkpoint in KRaft.
SnapshotHeaderRecord and SnapshotFooterRecord are not a concept in the bootstrap.checkpoint). Other KRaft control records include the kraft.version level and the starting voter set. QuorumController when it becomes leader. The active QuorumController attempts to write the bootstrap metadata records in a transaction if transactions are supported, or in a single atomic batch. From the perspective of KRaft, the contents of the bootstrap.checkpoint are "data" records, not control records. The bootstrap.checkpoint file is created by the kafka-storage tool. This file will be deprecated. The kafka-storage tool will not create this file any more and will instead write metadata record to the zero checkpoint in the cluster metadata partition.
The kafka-storage tool will delete the bootstrap.checkpoint file if it already exists. This is done to make sure that the metadata state is not contained in both bootstrap.checkpoint and the zero checkpoint.
This checkpoint file is also created by the kafka-storage tool in the __cluster-metadata-0 directory. Now, it will also contain the data that used to be contained in the bootstrap.checkpoint file.
When the controller handles RaftClient.Listener#handleLoadSnapshot if the snapshot id has an epoch of 0 and a base offset of 0, the controller will consider these records as the bootstrapping records. The controller will rewrite bootstrap records to the log if they have been successfully written in the past. If the controller doesn't load a snapshot at epoch 0 and offset 0, the controller will load the bootstrap.checkpoint and rewrite the bootstrapping record to the cluster metadata partition.
To be compatible with previous bootstrapping of Kafka, at controller activation the controller can be in the following states:
Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.