Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Backward-compatible: no existing metrics renamed or removed.

ModuleMetricTypePurpose
Broker — SharePartitionManagerTxnPendingRecordsCountGaugeCurrent count of records in TX_PENDING (primary health signal)
Broker — SharePartitionManagerTxnShareAcknowledgeRequestLatencyMsHistogramp99 latency of staging requests
Broker — SharePartitionManagerTxnPendingLockExpiredCountCounterAny non-zero = abandoned txns or missing markers (critical alert)
Broker — TransactionCoordinatorTransactionPartitionsCount (existing)GaugeReused; now includes __share_group_state-N entries
Producershare-ack-txn-send-rateMeterEOS-call throughput
Producershare-ack-txn-send-error-rateMeterStage-failure rate (drives retry loops)
Consumershare-group-metadata-fetch-rateMeterConfirms read-process-write loop is active
Total: 6 new + 1 reused = 7 metrics. Primary alert: TxnPendingRecordsCount > 0 sustained for > 60s.


Compatibility, Deprecation, and Migration Plan

  • Zero Breaking Changes: Current implicit and explicit modes remain the default and continue to function as-is.

  • Opt-in Requirement: Users must explicitly enable the new mode.

Test Plan

The verification strategy focuses on state machine integrity and fault tolerance under high-concurrency and failure scenarios.

...

Unit Tests: Validates state transitions (e.g., ACQUIRED to ACKNOWLEDGED on commit vs. AVAILABLE on abort), idempotency of operations, and transaction timeout/auto-abort logic.

...

Integration Tests: Focuses on end-to-end commit/abort flows, coordinator recovery, and multi-consumer behavior within a single group during network partitions.

...

System & Performance Tests: Benchmarks transactional vs. non-transactional modes and verifies exactly-once delivery.

Enablement and Rollout Plan

Feature Gating

opt-in, online-upgradable, and zero-disruption for existing workloads - No new feature flag introduced. Uses existing kafka-features.sh machinery.

Feature flagRequired levelWhy
share.version>= 2New level finalises KIP-1289; brokers below this advertise no apiKey <93>
transaction.version>= 2KIP-1289 is TV2-only (inherits auto-registration and proper epoch fencing)

Rolling Upgrade (Online; No Traffic Disruption)

Phase 1 — Software upgrade (rolling)
  Upgrade brokers one at a time to the binary containing KIP-1289.
  KIP-1289 code is dormant (share.version still 1).
  Standard Kafka rolling-restart

Phase 2 — Soak timing
  Verify cluster health

Phase 3 — Finalise feature
  kafka-features.sh upgrade --feature share.version --version 2
  - Online metadata propagation (update of  __cluster_metadata)
  - No restart, no rebalance, no leader change, no socket disruption
  - Brokers begin advertising apiKey 93

Phase 4 — Client onboarding
  Roll out applications calling sendShareAcknowledgementsToTransaction.
  Add the alerts (esp. TxnPendingRecordsCount) and relevant metrics in dashboard & observability.

Test Plan

The verification strategy focuses on state machine integrity and fault tolerance under high-concurrency and failure scenarios.

  • Unit Tests: Validates state transitions (e.g., ACQUIRED to ACKNOWLEDGED on commit vs. AVAILABLE on abort), idempotency of operations, and transaction timeout/auto-abort logic.

  • Integration Tests: Focuses on end-to-end commit/abort flows, coordinator recovery, and multi-consumer behavior within a single group during network partitions.

  • System & Performance Tests: Benchmarks transactional vs. non-transactional modes and verifies exactly-once delivery.

  • Chaos Tests: Simulates broker and coordinator crashes specifically during critical phases like PREPARE_COMMIT to ensure protocol durability.

Follow-up KIP (deferred)

•  Persistent TX_PENDING in __share_group_state for crash-resilient EOS on broker failover during staging (addresses Corner Cases).
•  Per-share-partition metric for TX_PENDING residency time — would add a histogram of "time spent in TX_PENDING" useful for diagnosing slow producers; can be added in the persistence KIP without compatibility concerns

...

.

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.