...

At that time, the AlterConfigs  API included all the configurations to be applied (usually after gathering all the current configurations first using DescribeConfigs  call). This has changed after KIP-339 was introduced. With IncrementalAlterConfigs , only configurations to be changed are including to the request and changes are calculated on the broker-side.

This current context Once incremental updates were adopted, only enables a very limited type of validations (e.g. config x  cannot not be y) as the current state is not included in the request, while most important validations have a dependency between configurations (e.g. if value of a=b  and c=d  then x  cannot be y). Even further, having some policies available only on creation but not when altering means the effort put to validate on creation is wasted on later changes.

...

• On Legacy Alter Config, the set of changes proposed (sum(c^i@n+1)) is the same as the new config (t@n+1) with the difference that null values are removed from the new config.
• On Incremental Alter Config, the set of changes proposed (sum(c^i@n+1)) is not the same as the new config (t@n+1), it only contains explicit changes to the config
• Implicit deletes are a set of configs created on legacy alter config, when no value is provided (not in t@n+1) but exists on the current config (t@n)
• Even though alter config policy receives the "requested" configurations, these have 2 different meanings depending on the API used to update configs.
  • When Legacy Alter Config, it means: requested changes (sum(c^i@n+1)) that is equal to new config state (t@n+1) including explicit deletes (sum(c^i@n+1 == null))
  • When Incremental Alter Config, it means: only requested changes (sum(c^i@n+1)) including explicit deletes (sum(c^i@n+1 == null)) but without any other config from current (!= t@n) or new status (!= t@n+1)
  • Plugin implementations do not know which one are they actually dealing with, and as incremental (new) API becomes broadly adopted, then configurations from the current status (t@n) not included in the request (sum(c^i@n+1)) are not considered.

Proposed Changes

Given the current limitations with the existing Alter Config policies, this KIP proposes to add a new version that explicitly considers the current (before alter) and updated (after alter) set of configurations.

• New broker configuration: alter.config.v2.policy.class.name 
• New interface: AlterConfigV2Policy  

Public Interfaces

1. Extend AlterConfigPolicy.RequestMetadata to include existing Add AlterConfigV2Policy  including before and after configurations:

public interface AlterConfigPolicyAlterConfigV2Policy extends Configurable, AutoCloseable {

    class RequestMetadata {

        private final ConfigResource resource;
        private final Map<String, String> proposedConfigsconfigsBefore;
        private final Map<String, String> resultingConfigsconfigsAfter;

        public RequestMetadata(ConfigResource resource, Map<String, String> configsBefore, Map<String, String> proposedConfigsconfigsAfter) {
            this.resource = resource;
            this.proposedConfigsconfigsBefore = proposedConfigsconfigsBefore;
            this.resultingConfigsconfigsAfter = Collections.emptyMap()configsAfter;
        }

		// ...

         public RequestMetadata(ConfigResource resource,   public Map<String, String> proposedConfigs, Map<String, String> resultingConfigsconfigsBefore() {
            this.resource = resource;
            this.proposedConfigs = proposedConfigs;
            this.resultingConfigs = resultingConfigsreturn configsBefore;
        }

		// ...

        public Map<String, String> resultingConfigsconfigsAfter() {
            return resultingConfigsconfigsAfter;
        }

        // ...

    void validate(RequestMetadata requestMetadata) throws PolicyViolationException;
}

Where:

• resultingConfigsconfigsBefore  includes the altered configurations: existing and proposed configurations merged.
• proposedConfigs includes the requested configurations, including null or empty values when configuration is requested to be deleted.
  • When using legacy alter configs, implicit deletes are not included as part of the proposed configs. See KAFKA-14195

For users using (legacy) alter configurations:

• They can still rely on proposed configs field to include all configurations.

For users using incremental alter configurations:

• They can use the new resulting configs field to apply the same policies as when creating topics or legacy alter configs.

Proposed Changes

Enable Managers to use the new property:

• ConfigurationControlManager 
• ZkAdminManager 

existing configurations are already available.

• current configurations
• configsAfter includes resulting configurations as if requested configurations are applied

Deleted configurations will either not appear or have null value on the configurations after alter, returning back to default value (if any).

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?

Existing policy implementations should not be affected, as new attribute won't be expected. New policy implementations will have to be deployed on brokers running newer versions including existing configurations.None, it's a new API.

• If we are changing behavior how will we phase out the older behavior?

Will be phased out naturally  by rolling out new versions with the latest APICurrent policy will be kept for compatibility. Later releases can consider deprecation.

• If we need special migration tools, describe them here.

...

• When will we remove the existing behavior?

Behavior is extended, not replacedOutside the scope of this KIP.

Test Plan

Extending existing Same as AlterConfigPolicy  tests.

...

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

Reuse

...

existing AlterConfigPolicy

Extending the configs, and its semantics, This would be a breaking change on behavior as existing plugins may be expecting only changing configurationsdeal with different meaning depending on the alter API used. To avoid dealing with existing issues/bugs a new API is proposed.