Table of Contents
Status
Current state: Under Discussion
Discussion thread:
JIRA:
Released:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
We have 2 methods in AdminClient
for updating config, alterConfigs
and incrementalAlterConfigs
, the former has many restrictions and has been deprecated from 2.3.0. However, It's still used in ConfigCommand
to update broker config and is resulting in a bug
, so I'm inclined to move it to Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key KAFKA-13788 incrementalAlterConfigs
and provide a flag to still use alterConfigs
for new client to interact with old servers. There are some other reasons for this change:
alterConfigs
has been deprecated and will be removed in a future release;- we are using
incrementalAlterConfigs
to change user/topic/client-metrics configs, it would be benefit to unify broker configs; incrementalAlterConfigs
is more convenient especially for updating configs of list data type, such as "leader.replication.throttled.replicas" , though we can’t subtract or append configs using kafka-configs.sh, we can make way for the future for appending/subtracting list properties by useincrementalAlterConfigs.
We are forced to pass all sensitive configs to update broker configs when using
alterConfigs
with the current cli tool because sensitive config values are never returned to the client, this result in KAFKA-13788, it must be resolved.
Note that I'm only changing the way we updating broker configs, user/topic/client-metrics configs are already being updated using incrementalAlterConfigs
Public Interfaces
kafka-configs.sh
we are changing the semantics of kafka-configs.sh without changing any command arguments.
- Existing sensitive broker properties no longer have to be explicitly specified on the command line if they're not being changed
- A small race condition is fixed where the broker config is updated by a separate operation in between when the CLI reads the existing broker config and writes the new broker config
- Usage of a new broker API that has been supported since version Adding new options --enable-incremental , which is used for client to connect to new servers after 2.3.0 when updating broker configs.
Proposed Changes
- ,
Proposed Changes
When updating broker config, instead of using Admin.alterConfigs, we will use Admin.incrementalAlterConfigs and fallback to use alterConfigs
automatically if incrementalAlterConfigs
is not supported, we are doing this heuristically instead of manually.
- We only do this when --enable-incremental is only makes sense when used together with (--alter) and (--broker <broker-id>)/(--entity-types brokers) are specified, or it will be ignored, for example when updating topic configs.
- --enable-incremental is only makes sense when used together with We only do this when (--bootstrap-server)/(bootstrap-controller) is specified, we are leaving zookeeper case unchanged.
- use AdminClientAdmin.
incrementalAlterConfigs
if --enable-incremental flag is specified when alter broker configs firstly, which will fail if the broker is before 2.3.0, and we will retry with Admin.alterConfigs() - AdminClient.alterConfigs -enable-incremental is deprecated and will be removed in remove In the future together with AdminClient.alterConfigs, and AdminClient.
incrementalAlterConfigs
will be the only choice choice then.
here is an example of changing the broker configs twice:
- set log.cleaner.threads=2
- set background.threads=1
in case 1 with old version , or in the new version without --enable-incremental, client, here are what happened:
- Use kafka-configs.sh to set log.cleaner.threads=2, the client will fetch all broker configs and get a empty properties {}, merge it with the delta and get {log.cleaner.threads=2}, send it to server using AdminClient.alterConfigs(), the server will persist it to metadata storage.
- Use kafka-configs.sh to set background.threads=1, the client will fetch all broker configs and get {log.cleaner.threads=2}, merge it with the delta and get {log.cleaner.threads=2, background.threads=1 }, send it to server using AdminClient.alterConfigs(), the server will persist it to metadata storage.
in new case with --enable-incremental2 with new version client, or when we removed the deprecated AdminClient.alterConfigs, here are what happened:
- Use kafka-configs.sh to set log.cleaner.threads=2, the client will send it to server using AdminClient.incrementalAlterConfigs(), the broker will merge the old snapshot( {} ) with delta and save the new snapshot( {log.cleaner.threads=2} ) to metadata storage.
- Use kafka-configs.sh to set background.threads=1, the client will send it to server using AdminClient.incrementalAlterConfigs(), the broker will merge the old snapshot( {log.cleaner.threads=2} ) with delta and got the new snapshot ({log.cleaner.threads=2, background.threads=1 }) to metadata storage.
in case 3 with new version client and old version server, we will try firstly as case 2, and fail with UnsupportedVersionException, and retry as case 3.
The result is expected to be the same for both way, except that we can avoid some issues for AdminClient.alterConfigs().
Compatibility, Deprecation, and Migration Plan
There will be a backward compatibility problem here since incrementalAlterConfigs
is added from 2.3.0, we add a --disable-incremental option for older client to do smooth migrationis no backward compatibility problems brought in because we are changing it in a compatible way.
Test Plan
new client tool with older server can be tested locally and using test cases.
Documentation Plan
There are some api changes and we should document about kafka-configs.sh change list.
Rejected Alternatives
- We fallback to using
alterConfigs
automatically ifincrementalAlterConfigs
is not supported. We need to unify the semantics if we want to do this heuristically, so we choose to give this privilege to users by adding --disable-incrementalAdding new option --enable-incremental to give this privilege to users, since we can do it heuristically without and side effect, and avoid adding a deprecated argument which should be removed in the future. - we make incrementalAlterConfigs the default way and add a "--disable-incremental" flag for old servers before Kafka 3.X, and remove it in Kafka 4.X. We are inclined to do move this incompatible process to 4.Xthis is the same with solution 1.
- We just forward all invocations of alterConfigs to incrementalAlterConfigs. Similar to first one, we need to unify the semantics, it's not recommend since alterConfigs is deprecated and we just leave it as it was.