THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| MBean name | Description | ||
|---|---|---|---|
| kafka.server:type=KafkaServer,name=MetadataType | An enumeration of: ZooKeeper (1) , Dual or KRaft (2). Each broker reports this. | ||
| kafka.controller:type=KafkaController,name=MetadataType | An enumeration of: ZooKeeper (1), KRaft (2), | KRaftor Dual (3). | Each brokerThe active controller reports this. |
| kafka.controller:type=KafkaController,name=Features,feature={feature},level={level} | The finalized set of features with their level as seen by the controller. Used to help operators see the cluster's current metadata.version | ||
| kafka.controller:type=KafkaController,name=MigrationState | An enumeration of the possible migration states the cluster can be in. This is only reported by the active controller. The "ZooKeeper" and "MigrationEligible" states are reported by the ZK controller, while the remaining states are reported by the KRaft controller. | ||
| kafka.controller:type=KafkaController,name=ZooKeeperWriteBehindLag | The amount of lag in records that ZooKeeper is behind relative to the highest committed record in the metadata log. This metric will only be reported by the active KRaft controller. | ||
| kafka.controller:type=KafkaController,name=ZooKeeperBlockingKRaftMillis | The number of milliseconds a write to KRaft has been blocked due to lagging ZooKeeper writes. This metric will only be reported by the active KRaft controller. |
...
All brokers must be running at least this MetadataVersion before the migration can begin. ZK brokers will specify their MetadataVersion using the inter.broker.protocol.version as usual. The KRaft controller will bootstrap with the same MetadataVersion (which is stored in the metadata log as a feature flag – see KIP-778: KRaft to KRaft Upgrades).
Configuration
A new “kafka.metadata.migration.enable” config will be added for the ZK broker and KRaft controller. Its default will be “false”. Setting this config to “true” on each broker is a prerequisite to starting the migration. Setting this to "true" on the KRaft controllers is the trigger for starting the migration (more on that below). Setting this to "true" (or "false") on a KRaft broker has no affect.
...
Here is a state machine description of the migration. There will likely be more internal states that the controller uses, but these four will be exposed as the MigrationState metric.
State | Enum | Description |
ZooKeeperMigrationIneligible | 1 | The brokers and controllers do not meet the migration criteria. The cluster is operating in ZooKeeper mode. The migration cannot yet begin. |
MigrationEligible | 2 | The cluster has been upgraded to a minimum software version and has set the necessary static configs |
MigrationReady | 3 | The KRaft quorum has been started |
MigrationActive | 4 | ZK state has been migrated, controller is in dual-write mode, brokers are being restarted in KRaft mode |
MigrationFinished | 5 | All of the brokers have been restarted in KRaft mode, controller still in dual-write mode |
KRaft | 6 | The cluster is in KRaft mode |
And a state machine diagram:
MigratingZkData | 2 | The controller is copying data from ZooKeeper into KRaft. |
DualWriteMetadata | 3 | The controller is in KRaft mode making dual writes to ZooKeeper. |
MigrationFinalized | 4 | The cluster has been migrated to KRaft mode. |
The active ZooKeeper controller always reports "MigrationIneligible" while the active KRaft controller reports the state corresponding to the state of the migration.
Preparing the Cluster
...