Page History

...

To make the upgrade completely compatible with current EOS transaction semantics, we need to be able to distinguish clients who are making progress on the same input source but using different transactional id. It is possible to have two different types of clients within the same consumer group. Imagine a case as a Kafka Streams applications, where half of the instances are using old task producer API, while the other half of them use new consumer group API. This fencing could be done by leveraging transactional offset commit protocol which contains a consumer group id and topic partitions. Group coordinator could build a reverse mapping from topic partition to [producer.id, generationId], and remember which producer has been contributing offset commits to each specific topic partition. In this way, when we upgrade to the new API, group coordinator will be actively checking this map upon receiving `TxnOffsetCommit`. If the stored `producer.id` doesn't match the one defined in request, or and the producer.generation id matches but in the epoch request field is either undefined or smaller than the one currently stored, coordinator would send out a `ConcurrentProducerCommitException` in the response to shutdown this conflict producer immediately. This ensures us a smooth upgrade without worrying about old pending transactions. Also this suggests that it is not recommended to have two types of clients running in the same application which makes the fencing much harder.

...