Status

Current state: Under Discussion Accepted

Discussion thread: here

Vote: here

JIRA: KAFKA-17200 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

    public static final String TOPICS_EXCLUDE_CONFIG_ALIAS = "topics.blacklist";
    private static final String TOPICS_EXCLUDE_DOC = "List of topics and/or regexes that should not be replicated.";
     public static final String TOPICS_EXCLUDE_DEFAULT = ".*[\\-\\.]internal, .*\\.replica, __.*";  

While the exclude list of the topic filter is configurable, the ReplicationPolicy interface cannot be configured in a way to enable replicating such topics. Currently, if a user already has business topics ending in for example ".internal" or "-internal", the only option is to implement a custom replication policy and override the isInternalTopic method. The goal of this proposal is to make this behaviour configurable.

...

• DefaultReplicationPolicy
• IdentityReplicationPolicy

The default exclude list of the DefaultTopicFilter will use the same, more specific pattern.

Proposed Changes

This KIP proposes to make the condition for internal topics more specific in the ReplicationPolicy:

...

public class DefaultReplicationPolicy implements ReplicationPolicy, Configurable { 
    ...      

    @Override
    public boolean isMM2InternalTopic(String topic) {
        return topic.startsWith("mm2") && topic.endsWith(internalSuffix()) || isCheckpointsTopic(topic);
    }    
}

The exclude list of the DefaultTopicFilter would also require some modification to reflect the new pattern:

public static final String TOPICS_EXCLUDE_DEFAULT = "mm2.*internal, .*\\.replica, __.*";

Compatibility, Deprecation, and Migration Plan

...