This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Under Discussion": Discarded, and superseded by KIP-215: Add topic regex support for Connect sinks

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: KAFKA-3073 here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

For now we provide topic names in a properties file as following:

...

If there are a few topics to deal with, It's ok to list all the topics in a comma-seperated way. But when you have to deal with hundreds or thousands of topics, it doesn't help. In those situations, we would like to use a wildcard expression to specify topics.

Public Interfaces

• client configuration file

Proposed Changes

• add an aditional option "topics.regex"
• a user can only specify one of "topics" or "topics.regex"
• if a user specifies "topics" in their config file, it works the same way as the current version. 
• if a user specifies "topics.regex", it can handle all the topics matching the regular expressions.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
  • N/A
• If we are changing behavior how will we phase out the older behavior?
  • N/A
• If we need special migration tools, describe them here.
  • Not needed
• When will we remove the existing behavior?
  • N/A

Rejected Alternatives

It is possible to update the interpretation of the existing option "topics" in a way that our users can specify reglar expressions there. But since there are some special characters in the regular expression that can be used in topic names, eg. "." and "-",  we need some workaround. Also this aproach has some effects on existing users. 

...