THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
group.instance.id | The unique identifier of the consumer instance provided by end user. Default value: empty string. |
Stream configs
stream.instance.id | The unique identifier of the stream instance provided by end user. Default value: empty string. |
Client side changes
The new `group.instance.id` config will be added to the join group request.
...
For the effectiveness of the KIP, consumer with `group.instance.id` set will not send leave group request when they go offline, which means we shall only rely on session.timeout to trigger group rebalance. It is because the proposed rebalance protocol will trigger rebalance with this intermittent in-and-out which is not ideal. In static membership we leverage the consumer group health management to client application such as K8. Therefore, it is also advised to make the session timeout large enough so that broker side will not trigger rebalance too frequently due to member come and go.
Kafka Streams change
KStream uses stream thread as consumer unit. For a stream instance configured with `num.threads` = 16, there would be 16 main consumers running on a single instance. We could actually borrow the idea for initializing client id for stream thread consumer to apply for `group.instance.id` generation.
For example if user specifies the client id, the stream consumer client id will be like: User client id + "-StreamThread-" + thread id + "-consumer". If user client id is not set, then we will use process id.
We could use a similar setting for defining `group.instance.id` on Stream. `stream.instance.id` (defined above) would be used as prefix, and for each thread consumer the formula will look like:
`group.instance.id` = `stream.instance.id` + "-" + thread id + "-consumer"
If the `stream.instance.id` is not set, we will not define `group.instance.id`. So the `stream.instance.id` is also used as the switch for using static membership on Stream.
Server behavior changes
On server side, broker will keep handling join group request <= v3 as before. The `member.id` generation and assignment is still coordinated by broker, and broker will maintain an in-memory mapping of {group.instance.id → member.id} to track member uniqueness. When receiving an known member's (A.K.A `group.instance.id` known) rejoin request, broker will return the cached assignment back to the member, without doing any rebalance.
...