Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The format sub-command in the kafka-storage.sh tool will include two new properties when producing meta.properties: directory.id and directory.ids. This sub-command already supports  formatting more than one log directory — by expecting a list of configured log.dirs  —  and "formatting" only the ones that need so. All configured log directories must be available for kafka-storage.sh format to run successfully.

meta.properties

To avoid issues when downgrading, the meta.properties version field will be kept at 1. Two new properties directory.id and directory.ids will be added to the meta.properties file in each log directory, including the metadata.log.dir. The first property, directory.id indicates the UUID for the log directory where the file is located, the second property, directory.ids  lists all the UUIDs for all the configured log directories. If the meta.properties  file doesn't exist for the metadata.log.dir  the Kafka node will fail to start. If the meta.properties  file exists but it doesn't contain these two properties a new one will be generated and the meta.properties  files will be updated. The kafka-storage.sh  tool will be extended to generate or update the two properties as described in the previous section.

...

The meta.properties  version field will stay set to 1, to allow for a downgrade after an upgrade on a non JBOD KRaft cluster.

Footnote

If an existing KRaft cluster is upgraded to the first version that writes these new fields is downgraded the meta.properties file needs to still be readable. There's currently a hard check on the version number which would fail for a new version number. 


The UUIDs for each log directory are automatically generated by the tool if there isn't one assigned already in an existing meta.properties  file.

...