You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

Status

Current state: Under Discussion

Discussion thread: https://lists.apache.org/thread/3gvsod6vrq5xb415yf0zs8cxry1lnfzf

JIRA: Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-133 introduced AlterConfigPolicy  to enforce validations when requesting configuration changes for Kafka entities (e.g. Topics), similar to how KIP-108 introduced policies for Topic creation.

These 2 policies differ on the following: CreateTopicPolicy  includes all configurations, which allow validations between related configurations; but AlterConfigPolicy  only includes the configuration passed on the request, but not the existing configurations or configurations to delete.

This force users to a very limited type of validation (e.g. config x  cannot not be y ), when most important validations have a dependency between configurations (e.g. if value of a=b  and c=d  then x  cannot be y ). Even more, having these configuration available on creation, but not on alter means the effort put to validate on creation is wasted when altering.

If this KIP is accepted, plugin implementations should be able to apply the same policies on creation and on alter config time.


There has been related KIPs that included this proposal:

This KIP is intended to reduce the scope of the proposal for extending AlterConfigPolicy, without adding new interfaces.

This KIP borrows parts of KIP-170 discussion. If KIP-201 is resurrected, this changes shouldn't increase complexity of the KIP as it is just another field to map and same migration should apply.

Public Interfaces

1. Extend AlterConfigPolicy.RequestMetadata to include existing configurations:

public interface AlterConfigPolicy extends Configurable, AutoCloseable {

    class RequestMetadata {

        private final ConfigResource resource;
        private final Map<String, String> proposedConfigs;
        private final Map<String, String> resultingConfigs;

        public RequestMetadata(ConfigResource resource, Map<String, String> proposedConfigs) {
            this.resource = resource;
            this.proposedConfigs = proposedConfigs;
            this.resultingConfigs = Collections.emptyMap();
        }

        public RequestMetadata(ConfigResource resource, Map<String, String> proposedConfigs, Map<String, String> resultingConfigs) {
            this.resource = resource;
            this.proposedConfigs = proposedConfigs;
            this.resultingConfigs = resultingConfigs;
        }

		// ...

        public Map<String, String> resultingConfigs() {
            return resultingConfigs;
        }

    // ...

    void validate(RequestMetadata requestMetadata) throws PolicyViolationException;
}

Where:

  • resultingConfigs  includes the altered configurations: existing and proposed configurations merged.
  • proposedConfigs includes the requested configurations, including null or empty values when configuration is requested to be deleted.

Proposed Changes

Enable Managers to use the new property:

  • ConfigurationControlManager 
  • ZkAdminManager 

existing configurations are already available.

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?

Existing policy implementations should be safe, as new attribute won't be expected. New policy implementations will have to be deployed on brokers running newer versions including existing configurations.

  • If we are changing behavior how will we phase out the older behavior?

Will be phased out naturally  by rolling out new versions with the latest API.

  • If we need special migration tools, describe them here.

No migration tools needed.

  • When will we remove the existing behavior?

Behavior is extended, not replaced.

Test Plan

Extending existing AlterConfigPolicy  tests.

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

Reuse configs value to include all resulting configurations

This would be a breaking change on behavior as existing plugins may be expecting only changing configurations.


  • No labels