...

NOTE: This part is drafted based on the assumption that KIP-31 and KIP-32 will be implemented in one patch.

The proposed protocol is not backward compatible. The migration plan are as below:

1. Bump up ProducerRequest and FetchRequest version to V2, which uses MessageAndOffset V1.
2. Create internal ApiVersion 0.9.0-1** which uses ProducerRequest V2 and FetchRequest V2.
3. Configure the broker to use ApiVersion 0.9.0 (ProduceRequest V1 and FetchRequest V1).
4. Do a rolling upgrade of the brokers to let the broker pick up the new code supporting ApiVersion 0.9.0-V1.
5. Bump up ApiVersion of broker to 0.9.0-1
6. Do a rolling bounce of the brokers to let the broker use FetchRequest V2 for replication.
7. Upgraded brokers support both ProducerRequest V2 and FetchRequest V2 which uses magic byte 1 for MessageAndOffset.
   
   1. When broker sees a producer request V1 (MessageAndOffset = V0), it will decompress the message, assign offsets using relative offsets and re-compress the message. i.e. upconvert the message format to MessageAndOffset V1.
      
   2. When broker sees a producer request V2 (MessageAndOffset = V1), it will decompress the message for verification, assign the offset to outer message and NOT do recompression. 
   3. When broker sees a fetch request V1 (MessageAndOffset = V0), because the data format on disk is MessageAndOffset V1, it will not use the zero-copy transfer, but read the message to memory, do down-conversion, then send fetch response V1.
   4. When broker sees a fetch request V2 (MessageAndOffset = V1), it will use zero-copy transfer to reply with fetch response V2.
      
8. Upgrade consumer to send FetchRequest V2.
9. Upgrade producer to send ProducerRequest V2.

...

During step 7 and step 8, the majority of the consumers may be still using consumers using MessageAndOffset V0, broker could consume more memory.

** We introduce internal ApiVersion here to help the user who are running on trunk to upgrade in the future. Otherwise the interim ApiVersion between two official releases will require users to downgrade ApiVersion then upgrade.

Rejected Alternatives

None

...