Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Status

Current State: Discussion
Discussion Thread: here
Jira: 

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-5669

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently very little checking is done against connector names, when creating a new connector in Kafka Connect. The only check that is performed is that no / is present in the name, other than that it is even possible to create a connector with an empty name that is then impossible to delete.
Additionally there are a large number of special characters that can be used in connector names but create issues afterwards, when they need to be used in the url for the rest call to change or delete the connector.
A pull request is ready for review that will deal with a few of the cases that cause issues, but this is more of a band aid than a proper fix for this issue, as I am sure there are a whole lot more characters out there that were missed and also create issues.

...

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4827

 

Proposed Change

Change the validation of connector names to use a whitelist of characters in order to ensure that no characters that create issues are allowed and enforce a maximum length for the connector name.

...

 

Code Block
a-z   A-Z   0-9   .   -   _   ~

 

New or changed public interfaces

There is no new public interface, however the behavior of the create connector api will change insomuch as some connector names that previously worked will now not be accepted anymore when creating new connectors. This may break deployment scripts for a few people, so we should definitely announce this as a breaking change, not sure if a full deprecation cycle makes sense, but since this will definitely be post 1.0 I guess it might be a good idea.


Migration plan and compatibility

Backward compatibility should be given, subject to testing.
Connectors with illegal names that have been deployed previously to the upgrade that incorporates are assumed to continue working (will need to be confirmed once code is available), and probably even changes to the configuration are possible since validators are only applied when creating connectors.

The migration path is to undeploy connectors and redeploy them with a legal name. This will not work for all connectors, as mentioned above, some characters cause issues, but these are broken anyway and will need to be removed manually by deleting them from the config topic in Kafka. This situation is no change from the current state.


Rejected alternatives

The obvious alternative is to use a blacklist of characters and reject connector names that contain one of these. However there is a good chance that we would spend a lot of time and jiras slowly refining this list while probably still missing
some extremely exotic characters. Also a case could be made that there is not really a point to using too many special characters in connector names, so I don't think there would be a large benefit to this approach.