This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Under Discussion

...

The bootstrap.checkpoint file is created by the kafka-storage tool. This file will be deprecated in a 4.x release and removed for existing clusters in the 5.0 release. The kafka-storage tool will not create this file any more and will instead write metadata record to the zero checkpoint in the cluster metadata partition.The kafka-storage tool will delete the bootstrap.checkpoint file if it already exists. This is done to make sure that the metadata state is not contained in both bootstrap.checkpoint and the zero checkpoint.

00000000000000000000-0000000000.checkpoint

This checkpoint file is also created by the kafka-storage tool in the __cluster-metadata-0  directory. Now, it will also contain the data that used to be contained in the bootstrap.checkpoint file. It is important to note that the bootstrap records must be less than 8MB of total size if transactions are not supported, since that is the maximum batch size in bytes supported by KRaft.

...

When the controller handles RaftClient.Listener#handleLoadSnapshot if the snapshot id has an epoch of 0 and a base offset of 0, the controller will consider these records as the bootstrapping records. The controller will rewrite bootstrap records to the log if they have haven't been successfully written in the past. If the controller doesn't load a snapshot at epoch 0 and offset 0, the controller will load the bootstrap.checkpoint and rewrite the bootstrapping record to the cluster metadata partition if they haven't been successfully written in the past.

Compatibility, Deprecation, and Migration Plan

...

1. bootstrap.checkpoint exist with metadata records and the zero checkpoint doesn't exists - In this case the controller will behave as it does today. The controller will be able to identify this case because RaftClient.Listener#handleLoadSnapshot won't ask to load the zero snapshot.
2. bootstrap.checkpoint exist with metadata records and the zero checkpoint exists but doesn't contain any metadata records - In this case the controller will behave as it does today. The controller will be able to identify this case because RaftClient.Listener#handleLoadSnapshot will ask to load the zero checkpoint but it will be empty, no metadata records.
3. bootstrap.checkpoint doesn't exist and the zero checkpoint exists with metadata records - In this case the controller will use the zero checkpoint's metadata records to and write them to the log in a transaction or single atomic batch like the controller does pre-KIP-1170.

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

1. bootstrap.checkpoint exist and the zero checkpoint exists with metadata records - This should not be possible from a formatting point of view but the active controller will handle this case the same as bullet 3 but with the addition of writing a WARN message to the controller log.

Test Plan

This feature will be tested using Java JUnit tests and system tests.

Rejected Alternatives

Not applicableIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.