DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
- New transient state TX_PENDING on a batch (orthogonal to RecordState); records in this state are NOT visible to peers, NOT redeliverable, lock NOT eligible for expiry.
- On WriteTxnMarkers commit: TX_PENDING(ACCEPT) → ACKNOWLEDGED; same for RELEASE and REJECT.
- On WriteTxnMarkers abort: TX_PENDING(*) → back to ACQUIRED (lock continues; consumer can retry the work).
...
Pseudo code
| Code Block |
|---|
// Kafka-as-destination path (true EOS via TxnShareAcknowledge):
batch boundary (or per-record, depending on tx granularity):
producer.beginTransaction()
for each (record, output) in batch:
producer.send(destinationTopic, output)
producer.sendShareAcknowledgementsToTransaction(
ackMap, // record→ACCEPT for processed records
shareConsumer.shareGroupMetadata()
)
producer.commitTransaction()
// No separate consumer.acknowledge() — the ACK is in the transaction.
-------------------
// Pure-external-sink path (at-least-once + idempotent destination):[this KIP is not changin anything of this flow]
worker thread per record:
output = process(record)
externalSink.write(output, idempotencyKey) // user task does this
consumer.acknowledgeAsync(record, ACCEPT)
.thenAccept(_ -> markComplete(record)) |
3. Compatibility, Deprecation, and Migration Plan
Zero Breaking Changes: Current
implicitandexplicitmodes remain the default and continue to function as-is.Opt-in Requirement: Users must explicitly enable the new mode.
...
4. Test Plan
The verification strategy focuses on state machine integrity and fault tolerance under high-concurrency and failure scenarios.
...