DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: Under Discussion
...
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
KIP-932 introduced Share Groups to Kafka, enabling cooperative consumption with queueing semantics. Currently, the behavior of share groups is governed by several broker-level configurations. While some configurations can be dynamically updated, some limits are currently static and applied globally to all share groups on a broker. This KIP aims to address this limitation by enabling dynamic, group-level configuration for these essential parameters.
Public Interfaces
The following table summarizes the configuration parameters for Share Groups. It indicates which parameters are currently supported as dynamic group configurations and which are introduced by this KIP.
| Broker Configuration Key |
|---|
| Description | Group Level Dynamic Config Support | |
|---|---|---|
| group.share.session.timeout.ms | The timeout to detect client failures when using the share group protocol. | Yes |
| group.share.heartbeat.interval.ms | The heartbeat interval given to the members of a share group. | Yes |
| group.share.record.lock.duration.ms | The record acquisition lock duration in milliseconds for share groups. | Yes |
| group.share.partition.max.record.locks | Share-group record lock limit per share-partition. | No |
| group.share.delivery.count.limit | The maximum number of delivery attempts for a record delivered to a share group. | No |
| group.share.max.size | The maximum number of members that a single share group can accommodate. | No |
Proposed Changes
We propose adding the following configurations to extend the list of supported dynamic configs for groupsdynamic group configuration mechanism to include share.partition.max.record.locks, share.delivery.count.limit, and share.max.size. This allows these parameters to be dynamically configured and tuned for each share groups at runtime.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?
Test Plan
Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
Rejected Alternatives
This change is fully backward compatible. Older broker versions will continue to apply only the broker-level static configurations. After upgrading to the new version, users can dynamically update the group-level configurations to override these values.
Test Plan
Unit tests and integration tests will be added.
Rejected Alternatives
NoneIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.