Current state: Accepted
Discussion thread: https://lists.apache.org/thread/xd28mgqy75stgsvp6qybzpljzflkqcsy
JIRA: https://issues.apache.org/jira/browse/KAFKA-16181
Released:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
We have 2 methods in AdminClient
for updating config, alterConfigs
and incrementalAlterConfigs
, the former has many restrictions and has been deprecated from 2.3.0. However, It's still used in ConfigCommand
to update broker config and is resulting in a bug , so I'm inclined to move it to incrementalAlterConfigs
and fallback to use alterConfigs if the server is under 2.3.0. There are some other reasons for this change:
alterConfigs
has been deprecated and will be removed in a future release;incrementalAlterConfigs
to change user/topic/client-metrics configs, it would be benefit to unify broker configs;incrementalAlterConfigs
is more convenient especially for updating configs of list data type, such as "leader.replication.throttled.replicas", though we can’t subtract or append configs using kafka-configs.sh, we can make way for the future for appending/subtracting list properties by use incrementalAlterConfigs.
We are forced to pass all sensitive configs to update broker configs when using alterConfigs
with the current cli tool because sensitive config values are never returned to the client, this result in KAFKA-13788, it must be resolved.
Note that I'm only changing the way we updating broker configs, user/topic/client-metrics configs are already being updated using incrementalAlterConfigs
we are changing the semantics of kafka-configs.sh without changing any command arguments.
When updating broker config, instead of using Admin.alterConfigs, we will use Admin.incrementalAlterConfigs and fallback to use alterConfigs
automatically if incrementalAlterConfigs
is not supported, we are doing this heuristically instead of manually.
incrementalAlterConfigs
firstly, which will fail if the broker is before 2.3.0, and we will retry with Admin.alterConfigs()incrementalAlterConfigs
will be the only choice then.here is an example of changing the broker configs twice:
in case 1 with old version client, here are what happened:
in case 2 with new version client, or when we removed the deprecated AdminClient.alterConfigs, here are what happened:
in case 3 with new version client and old version server, we will try firstly as case 2, and fail with UnsupportedVersionException, and retry as case 3.
The result is expected to be the same for both way, except that we can avoid some issues for AdminClient.alterConfigs().
There is no backward compatibility problems brought in because we are changing it in a compatible way.
We should pay attention to KAFKA-10140 in which we have a problem updating jmx configs using `incrementalAlterConfigs` but we can still update jmx configs using `alterConfigs`, however, the problem only appear when append/subtract list type config which is not supported by kafka-configs.sh, so it will not influence this KIP, and we leave it in another one when we try to support append/subtract in kafka-configs.sh.
new client tool with older server can be tested locally and using test cases.
There are some api changes and we should document about kafka-configs.sh change list.