This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Under Discussion [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: here

JIRA: KAFKA-13478

...

The following method (inspired by the existing org.apache.kafka.connect.connector.Connector.validate(Map<String, String>) method) will be added to the interface org.apache.kafka.connect.transforms.Transformation:    default Config

    /**
     * Validate the SMT configuration values against configuration definitions.
     * @param smtConfigs the provided configuration values
     * @return The updated configuration information given the current configuration values
     *
     * @since 3.2
     */
    default Config validate(Map<String, String> smtConfigs) {

        ConfigDef configDef = config();

...

The same method will be added to the org.apache.kafka.connect.transforms.predicates.Predicate interface, org.apache.kafka.connect.storage.Converter, and org.apache.kafka.connect.storage.HeaderConverter interfaces.

As this is a default method, this change is generally backwards compatible, i.e. existing Transformation, Predicate, Converter, and Predicate HeaderConverter implementations will continue to work as-is.

The method org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(Map<String, String>, boolean) will be adjusted to:

• Determine the list of configured SMT(s) and predicates SMTs/predicates/converters from the connector configuration passed for validation
• Instantiate each configured SMTSMTs/predicates/predicate converters and invoke its validate() method with the subset of the connector configuration of that particular SMTSMTs/predicates/predicateconverters
  
• Add the validation result(s) to the existing result type structure

This will make sure that the SMT SMTs/predicates/converters attributes are validated upon explicit calls to the validate REST method as well as when a connector gets registered or updated.

...

If a user had called the REST API’s validation method with an invalid SMT configuration in the past, this would have been ignored, whereas in the future, this call will yield the corresponding validation errors. While this is a behavioral change, it is a desirable one, as users will benefit from the early validation feedback provided through this change. It seems highly unlikely that a user intentionally relied on SMT configuration failures to not be reported.

If an existing Transformation or Predicate, Predicate, Converter, or HeaderConverter  implementation declares a method with the exact same signature as the proposed validate() method, it will be invoked by AbstractHerder.validateConnectorConfig(), which may be surprising. Impact of this should be low, apart from the fact that the same validation routine will potentially be invoked multiple times.

...

Apart from the following exception, the proposed change is binary compatible, i.e. an existing Transformation, Predicate, Converter, or Predicate HeaderConverter implementation class file compiled against an old version of the API can be used with the new API version without problems.

...

If an existing Transformation, Predicate, Converter, or PredicateHeaderConverter implementation declares

• a method validate(Map<String, String> connectorConfigs) with a return type other than Config, or
• a method Config validate(Map<String, String> connectorConfigs) with another visibility than public,

...