Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

public class KafkaShareConsumer<K, V> {

    /**

   * Acknowledge records within the provided session.

     */

    public void acknowledgeTransactionally(

        TransactionSession session,

        Map<TopicPartition, Set<Long>> acknowledgments

    );

}
```

...

3. Proposed Changes

3.1 Architecture: Before and After

...

Dedicated Connection - Maintains its own connection to the coordinator broker.


4. Use Cases

4.1 Apache Flink: Lightweight Transaction Completion

...

Architectural comparison:



...

5. Compatibility, Deprecation, and Migration Plan

5.1 Full Backward Compatibility

...

No deprecations. The KafkaProducer convenience methods remain the recommended API for simple produce-and-commit patterns. TransactionSession is for advanced use cases: 2PC, cross-entity transactions, external coordinators.

...

6. Security

6.1 Authorization

TransactionSession requires the same ACLs as the current producer transaction API:

...

The session identity is not multiplied -- there is one session, one producerId, one epoch, regardless of how many clients use it

...

7. Test Plan

...

7.1 Unit Tests

TestDescription
Session LifecycleVerify transitions: UNINITIALIZED INITIALIZING READY IN_TRANSACTION COMMITTING READY.
Session ResumeVerify resume() starts in IN_TRANSACTION and can execute commitTransaction().
Heartbeat LogicVerify heartbeat thread lifecycle based on transaction.session.timeout.ms.
Producer FencingEnsure KafkaProducer throws IllegalStateException on lifecycle calls when using an external session.
Backward CompatibilityVerify standard KafkaProducer transaction methods work via internal session delegation.
Epoch FencingVerify that a new session with the same transactional.id correctly fences the older session.
Identity AccessorsEnsure producerId(), producerEpoch(), and transactionalId() are accurate post-initialization.

...

7.2 Integration Tests

Producer + external session E2E

Resume and commit from different process

...

7.3 Compatibility Tests

TestDescription
Legacy ProducerRun existing test suite on KafkaProducer without external sessions to ensure zero regression.
Mixed ModeRun internal and external sessions concurrently on the same cluster to verify no interference.

9. Reference 

KIP-98 - Exactly Once Delivery and Transactional Messaging

...