Table of Contents |
---|
Status
Current state: Under Discussion Accepted
Discussion thread: here
JIRA:
Jira | ||||||
---|---|---|---|---|---|---|
|
...
Code Block | ||
---|---|---|
| ||
case class Broker(id: Int, endPoints: Map[SecurityProtocol, EndPoint], rack: Option[String] = None) |
rack will be an optional field of version 3 JSON schema for a broker. The API ZkUtils.registerBrokerInZk will be updated to increment the JSON version.
...
The version will be incremented to 2 from 1 and rack will be included. For version 0, only broker ID, host and port will be serialized. For version 1 onwardsand 2, the complete Broker object will be serialized. This is done by calling Broker.writeTo and Broker.readFrom. Therefore, for version 2 the rack information will be automatically handled and the the serialization format is the same as the above.
...
Code Block |
---|
CorrelationID BrokerEndPoints TopicsMetaData BrokerEndPoints => NumberOfBrokers [BrokerEndPoint] BrokerEndPoint => BrokerID Host Port Rack // Rack is added BrokerID => int32 Host => string Port => int32 Rack => string |
...
- Rack will be included in inter-broker protocol. Therefore, rolling upgrade requires these steps:
- Update server.properties file on all brokers and add the following property: inter.broker.protocol.version=<old version> (for example, 0.9.0).X
- Upgrade the brokers. This can be done a broker at a time by simply bringing it down, updating the code, and restarting it.
- Once the entire cluster is upgraded, bump the protocol version by editing inter.broker.protocol.version and setting it to new version, for example, 0.910.10.
- Restart the brokers one by one for the new protocol version to take effect
- Rack property will be included in version 3 broker JSON schema and version 2 of UpdateMetaDataRequestUpdateMetadataRequest. Controller will include version specific broker information in its wire format so that broker with old version can still interoperate with broker with new version and rack.
Due to a bug introduced in 0.9.0.0 in ZkUtils.getBrokerInfo(), old clients will throw an exception when it sees the broker JSON version is not 1 or 2. Therefore, a minor release 0.9.0.1 is required to fix the problem first so that old clients can parse future version of broker JSON in ZooKeeper. That means 0.9.0.0 clients must be upgraded to 0.9.0.1 before 0.10.0 upgrade can start. In addition, since ZkUtils.getBrokerInfo() is also used by broker, version specific code has to be used when registering broker with ZooKeeper. This is outlined as follows:
When inter.broker.protocol.version is set to 0.9.0.X, the broker will write
the broker info in ZK using version 2, ignoring the rack info.When inter.broker.protocol.version is set to 0.10.0, the broker will write
the broker info in ZK using version 3, including the rack info.If one follows the upgrade process, after the 2nd round of rolling bounces,
every broker is capable of parsing version 3 of broker info in ZK. This is
when the rack-aware feature will be used
.