Page History

...

Storage will have to be formatted — using the storage format tool — not only when a log directory is added, but also when one is removed from configuration.

meta.properties

Two new properties storage.id and storage.ids will be added to the meta.properties file in each log directory. The first property, storage.id indicates the UUID for the log directory where the file is located, the second property, storage.ids  lists all the UUIDs for all the configured log directories. If the meta.properties  file doesn't exist for the cluster metadata partition the Kafka node will fail to start. If the meta.properties file exist but it doesn't contain these two properties a new one will be generated and the meta.properties  files will be re-written. The kafka-storage CLI tool will be extended to generate and write the two properties when the format command is used.

Metadata records

RegisterBrokerRecord will and BrokerRegistrationChangeRecord both have two new fields:

...

When the broker starts up and initializes the LogManager, it will load the UUID for each log directory (storage.id ) and the list of all log directory UUIDs (storage.ids), by reading the meta.properties file at the root of the each log directory. If different meta.properties files indicate different storage.ids the broker will fail at startup. It will diff the UUIDs with the list of all UUIDs to obtain the list of UUIDs for offline log directories. The lists of both online and offline log directory UUIDs are sent along in the broker registration request to the controller.

...