Page History

...

This is a server-client integrated change, and it's required to upgrade the broker first with `inter.broker.protocol.version` to the latest. Any produce request with higher version will automatically get fenced , because of no support. Because we will use the same transaction different transactional id for different same topic partitions at the same time (breaking EOS), while losing transaction status tracking once we switched coordinator location  we lose old transaction information if coordinator location gets switched. Once server is upgraded, an existing application could opt to use this new feature by upgrading to latest version and either change producer syntax (non-stream app) or set `consumer.group.aware.transaction` to true (stream app). Since the transaction.id will be reused and different from previous one and transaction coordinator might be changed, this upgrade is not guaranteed to retain consistent commit log on the new group coordinator located broker.

Right now we have  `AddOffsetsToTxnRequest` which contains a consumer group id. By leveraging this information, we could build a reverse mapping from topic partition to producer.id. Thus when we upgrade to the new API, group coordinator will be looking at this reverse mapping and send out abort transaction requests to all the affected transaction coordinators, which ensures us a smooth upgrade without worrying about old pending transactions. To remain compatible with current semantic during upgrade, we would proactively reject old version producer if it tries to init transaction towards a topic partition that is owned by some other producer. This may potentially requires a new Request type to let group coordinator inform other transaction coordinator to fence old producer and let them crash immediately.

...