Page History

...

1. Bump up ProducerRequest and FetchRequest version to V2, which uses MessageAndOffset V1.
2. Create internal ApiVersion 0.9.0-V1 1 which uses ProducerRequest V2 and FetchRequest V2.
3. Configure the broker to use ApiVersion 0.9.0 (ProduceRequest V1 and FetchRequest V1).
4. Do a rolling upgrade of the brokers to let the broker pick up the new code supporting ApiVersion 0.9.0-V1.
5. Bump up ApiVersion of broker to 0.9.0-V11
6. Do a rolling bounce of the brokers to let the broker use FetchRequest V2 for replication.
7. Upgraded brokers support both ProducerRequest V2 and FetchRequest V2 which uses magic byte 1 for MessageAndOffset.
   
   1. When broker sees a producer request V1 (MessageAndOffset = V0), it will decompress the message, assign offsets using relative offsets and re-compress the message. i.e. upconvert the message format to MessageAndOffset V1.
      
   2. When broker sees a producer request V2 (MessageAndOffset = V1), it will decompress the message for verification, assign the offset to outer message and NOT do recompression. 
   3. When broker sees a fetch request V1 (MessageAndOffset = V0), because the data format on disk is MessageAndOffset V1, it will not use the zero-copy transfer, but read the message to memory, do down-conversion, then send fetch response V1.
   4. When broker sees a fetch request V2 (MessageAndOffset = V1), it will use zero-copy transfer to reply with fetch response V2.
      
8. Upgrade consumer to send FetchRequest V2.
9. Upgrade producer to send ProducerRequest V2.

...