Status

Current state: Under Discussion

Discussion thread: here

JIRA: here 

Motivation

Since KIP-690: Add additional configuration to control MirrorMaker 2 internal topics naming convention, the replication.policy.separator has been used to control the name of internal topics for DefaultReplicationPolicy, causing backward compatibility issues if users set a customized replication.policy.separator. This impacted 3.1, 3.2, 3.3, 3.4, 3.5 versions. 

Currently, the only way to work around this is either by providing a new version of ReplicationPolicy (which can optionally subclass the DefaultReplicationPolicy class) that overrides the ReplicationPolicy.offsetSyncsTopic and ReplicationPolicy.checkpointsTopic methods to use old topics if users still want to use the old internal topics, or by deleting the old internal topics and letting MM2 set up the new ones.

In this KIP, we are proposing to add replication.policy.internal.topic.separator.enabled to allow customers to control this using configurations instead of providing a new implementation.

Public Interfaces

Proposed Changes

public class DefaultReplicationPolicy implements ReplicationPolicy, Configurable {

	private Boolean isInternalTopicSepratorEnabled;
	
    @Override
    public void configure(Map<String, ?> props) {
        if (props.containsKey(INTERNAL_TOPIC_SEPRATOR_ENABLED_CONFIG)) {
            isInternalTopicSepratorEnabled = (Boolean) props.get(INTERNAL_TOPIC_SEPRATOR_ENABLED_CONFIG);
        }
    }

    private String internalSuffix() {
        return isInternalTopicSepratorEnabled ? separator + "internal" : ".internal";
    }

}

Backporting plan

• The KIP need to be released as part of bugfix release to the last 3 versions
• The KIP need to be backported into 3.1 and 3.2 (any other version reached EOL post KIP-690).  

Compatibility, Deprecation, and Migration Plan

Users who upgrade from Kafka < 3.1 and wish to disable controlling internal topics using the separator will need to set `replication.policy.internal.topic.separator.enabled` to false. Any users upgrading from > 3.1 don't need to do anything.