DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
None of these fields are producer-specific. The naming is historical -- producerId and producerEpoch could equally be called sessionId and sessionEpoch. The wire protocol RPCs (InitProducerId, AddPartitionsToTxn, EndTxn, TxnHeartbeat) carry only these identity fields. They do not carry a "client type" discriminator.
What KIP-939 Solved
KIP-939 enabled Kafka to act as a formal participant in Two-Phase Commit (2PC) transactions by:
Adding Recovery APIs: Introduced
prepareTransaction()andcompleteTransaction(), allowing external systems to coordinate Kafka's commit/abort status.Resilient Resumption: Allowed new producers to resume a "prepared" transaction after a crash without automatically aborting it.
Removing Timeouts: Provided an
enable2Pcmode that sets transaction timeouts to infinity, preventing premature aborts during external coordination.
The Unsolved Problem
While KIP-939 added the right tools, it put them in the wrong place:
Wrong Abstraction: Transaction methods (like preparing and completing) are forced into the
KafkaProducerclass, even though they don't involve producing records.Heavyweight Bloat: To simply commit a transaction, you are forced to instantiate a full
KafkaProducerwith its entire infrastructure (buffer pools, sender threads, serializers).Inaccessible Logic: The core "brain" of transactions—the state machine and coordinator logic—is buried in internal producer packages, making it impossible for other clients (like consumers or custom coordinators) to use transactions independently.
1.2 Entities Already Need Transaction Participation
...
| Entity | Role | Current Workaround | Core Pain Point |
| KafkaProducer | Owner | Native API | Transaction logic is monolithic and coupled to the produce path. |
| Kafka Streams | EOS Processor | Wraps Producer | Lifecycle is forced to match record batching. |
| Connect Source | EOS Ingest | Wraps Producer | Boundary management is coupled to producer lifecycle. |
| Connect Sink | KIP-1302 | N/A | No way to share transaction identity between consumer and producer. |
| Flink / 2PC | External Committer | Reflection | Fragile; manually forces state into TransactionManager internals. |
| Share Consumer | KIP-1289 | N/A | Must "borrow" Producer IDs without a proper client abstraction. |
...
2. Public Interfaces
2.1 New Class: TransactionSession
Package: org.apache.kafka.clients.transaction
TransactionSession is a lightweight, thread-safe object that holds transaction identity and provides lifecycle operations. It does NOT produce records, consume records, or manage batching/serialization. It owns only the coordinator interaction.client focused strictly on transaction coordinator interaction.
It decouples lifecycle management from record production.
```java
public
...
class
...
TransactionSession
...
implements
...
Closeable
...
{
public TransactionSession(Map<String, Object> configs);
public static TransactionSession resume(
String transactionalId,
long producerId,
short producerEpoch,
Map<String, Object> configs
);
// --- Lifecycle ---
public void initialize();
public void beginTransaction();
public PreparedTxnState prepareTransaction();
public void completeTransaction(PreparedTxnState preparedTxnState);
public void commitTransaction();
public void abortTransaction();
// --- Identity ---
public String transactionalId();
// --- Participation ---
public void addPartitionsToTransaction(Collection<TopicPartition> partitions);
public void sendOffsetsToTransaction(
Map<TopicPartition, OffsetAndMetadata> offsets,
ConsumerGroupMetadata groupMetadata
);
public void addShareAcksToTransaction(
String groupId,
Collection<ShareAcknowledgment> acknowledgments
);
// --- Liveness ---
public void heartbeat();
@Override
public void close();
}
```
2.2 Configuration
TransactionSession accepts a subset of existing producer configs plus one new config:
...