...

Instead, records from a partition are acquired by any available consumer in the group. After processing, the consumer acknowledges the record (ACCEPT, RELEASE, ARCHIEVED, or REJECT).

This provides:

- Elastic Scaling: Decouples parallelism from partition count,
- No Head-of-Line Blocking: Supports unordered message processing; if a task slows down, records time out and are redelivered to available workers.
- Seamless Scaling: Eliminates "rebalance storms" by removing the partition assignment protocol, ensuring zero downtime during task membership changes.

Note: The share groups are only suitable for connectors with idempotent, order-independent processing.

1.3 Kafka Connect as the Natural Integration Point

Kafka Connect is the standard framework for building data pipelines into and out of Kafka. Integrating Share Groups into Connect's sink connector

runtime gives every existing sink connector access to queue semantics with a configuration change only -- no connector code modifications required.

1.4 Delivery Semantics

...

At-least-once is the initial target. Exactly-once requires KIP-1289 (Transactional Acknowledgments for Share Groups) to be implemented and is described as a future phase.

1.5 Error Handling / Delivery Semantics

This KIP integrates with KIP-1191. In Share Group mode, Connect uses `AcknowledgeType.REJECT` for fatal errors and relies on the broker-side DLQ configured by KIP-1191. Retriable failures use `AcknowledgeType.RELEASE`. If a DLQ is configured for the share group, the broker is the single source of DLQ records; Connect does not emit its own DLQ records in this mode.

EO Constraints

For exactly-once, records with pending transactional acknowledgments must not be re-delivered while the transaction is open; KIP-1289 must suppress or renew acquisition locks until commit/abort. Operationally, `share.acquisition.lock.timeout.ms` must exceed worst-case `task.put()` plus transaction commit latency, otherwise duplicates are possible even with EOS.

2. Scope

2.1 In Scope

...

(What we are building)

• New Task Type: Introducing WorkerShareSinkTask to handle Share Group logic without changing existing connector code.

• Flexible Activation: Toggle queue semantics globally or per-connector via consumer.override.group.protocol=

...

• share.

...

• Delivery Guarantees: * At-least-once

...

• : Standard support for all

...

• sink types.

...

• • Exactly-once

...

• • : Supported for same-cluster Kafka-to-

...

• • Kafka paths (requires KIP-1289

...

• • ).

...

• Observability: New Share

...

• Group metrics (acquisition, release, and rejection rates) integrated into the existing

...

• sink-task-

...

• metrics group.

2.2 Out of Scope

...

(Future/Separate efforts)

• Source Connectors: Share Group support

...

• is currently for Sinks only (Source support and MirrorMaker 2 are excluded).

• Cross-Cluster EOS: Exactly-once delivery between different Kafka clusters is not supported in this phase.

• API Changes: No modifications will be made to the public SinkTask Java API or individual connector codebases.

• Complex Transactions: External 2PC coordinators and cross-cluster transactional protocols are not addressed.

3. Public Interfaces

3.1 New Configuration Properties

...