DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Backward-compatible: no existing metrics renamed or removed.
| Module | Metric | Type | Purpose |
|---|---|---|---|
| Broker — SharePartitionManager | TxnPendingRecordsCount | Gauge | Current count of records in TX_PENDING (primary health signal) |
| Broker — SharePartitionManager | TxnShareAcknowledgeRequestLatencyMs | Histogram | p99 latency of staging requests |
| Broker — SharePartitionManager | TxnPendingLockExpiredCount | Counter | Any non-zero = abandoned txns or missing markers (critical alert) |
| Broker — TransactionCoordinator | TransactionPartitionsCount (existing) | Gauge | Reused; now includes __share_group_state-N entries |
| Producer | share-ack-txn-send-rate | Meter | EOS-call throughput |
| Producer | share-ack-txn-send-error-rate | Meter | Stage-failure rate (drives retry loops) |
| Consumer | share-group-metadata-fetch-rate | Meter | Confirms read-process-write loop is active |
| Total: 6 new + 1 reused = 7 metrics. Primary alert: TxnPendingRecordsCount > 0 sustained for > 60s. |
Compatibility, Deprecation, and Migration Plan
Zero Breaking Changes: Current
implicitandexplicitmodes remain the default and continue to function as-is.Opt-in Requirement: Users must explicitly enable the new mode.
Test Plan
The verification strategy focuses on state machine integrity and fault tolerance under high-concurrency and failure scenarios.
...
Unit Tests: Validates state transitions (e.g., ACQUIRED to ACKNOWLEDGED on commit vs. AVAILABLE on abort), idempotency of operations, and transaction timeout/auto-abort logic.
...
Integration Tests: Focuses on end-to-end commit/abort flows, coordinator recovery, and multi-consumer behavior within a single group during network partitions.
...
System & Performance Tests: Benchmarks transactional vs. non-transactional modes and verifies exactly-once delivery.
Enablement and Rollout Plan
Feature Gating
opt-in, online-upgradable, and zero-disruption for existing workloads - No new feature flag introduced. Uses existing kafka-features.sh machinery.
| Feature flag | Required level | Why |
|---|---|---|
| share.version | >= 2 | New level finalises KIP-1289; brokers below this advertise no apiKey <93> |
| transaction.version | >= 2 | KIP-1289 is TV2-only (inherits auto-registration and proper epoch fencing) |
Rolling Upgrade (Online; No Traffic Disruption)
Phase 1 — Software upgrade (rolling)
Upgrade brokers one at a time to the binary containing KIP-1289.
KIP-1289 code is dormant (share.version still 1).
Standard Kafka rolling-restart
Phase 2 — Soak timing
Verify cluster health
Phase 3 — Finalise feature
kafka-features.sh upgrade --feature share.version --version 2
- Online metadata propagation (update of __cluster_metadata)
- No restart, no rebalance, no leader change, no socket disruption
- Brokers begin advertising apiKey 93
Phase 4 — Client onboarding
Roll out applications calling sendShareAcknowledgementsToTransaction.
Add the alerts (esp. TxnPendingRecordsCount) and relevant metrics in dashboard & observability.
Test Plan
The verification strategy focuses on state machine integrity and fault tolerance under high-concurrency and failure scenarios.
Unit Tests: Validates state transitions (e.g.,
ACQUIREDtoACKNOWLEDGEDon commit vs.AVAILABLEon abort), idempotency of operations, and transaction timeout/auto-abort logic.Integration Tests: Focuses on end-to-end commit/abort flows, coordinator recovery, and multi-consumer behavior within a single group during network partitions.
System & Performance Tests: Benchmarks transactional vs. non-transactional modes and verifies exactly-once delivery.
Chaos Tests: Simulates broker and coordinator crashes specifically during critical phases like
PREPARE_COMMITto ensure protocol durability.
Follow-up KIP (deferred)
• Persistent TX_PENDING in __share_group_state for crash-resilient EOS on broker failover during staging (addresses Corner Cases).
• Per-share-partition metric for TX_PENDING residency time — would add a histogram of "time spent in TX_PENDING" useful for diagnosing slow producers; can be added in the persistence KIP without compatibility concerns
...
.
Rejected Alternatives
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.