Table of Contents |
---|
Status
Current state: Under Discussion Adopted
Discussion thread: here
JIRA: here
...
Since KIP-690: Add additional configuration to control MirrorMaker 2 internal topics naming convention, the replication.policy.separator
has been used to control the name of internal topics for DefaultReplicationPolicy
, causing backward compatibility issues if with offset syncs and checkpoint topics when users set a customized replication.policy.separator
. This impacted 3.1, 3.2, 3.3, 3.4, 3.5 versions.
...
In this KIP, we are proposing to add replication.policy.internal.topic.separator.enabled
to allow customers to control this using configurations instead of providing a new implementation for both checkpoints and offset sync topics. The heartbeat topic isn't controlled by the separator at all.
Public Interfaces
Property | Default value | Description |
---|---|---|
replication.policy.internal.topic.separator.enabled | true | whether or not to use replication.policy.separator to control internal topic names |
...
Code Block | ||
---|---|---|
| ||
public class DefaultReplicationPolicy implements ReplicationPolicy, Configurable { private Boolean isInternalTopicSepratorEnabledisInternalTopicSeparatorEnabled; @Override public void configure(Map<String, ?> props) { if (props.containsKey(INTERNAL_TOPIC_SEPRATORSEPARATOR_ENABLED_CONFIG)) { isInternalTopicSepratorEnabled = (Boolean) props.get(INTERNAL_TOPIC_SEPRATOR_ENABLED_CONFIG); } } private String internalSuffixinternalSeparator() { return isInternalTopicSepratorEnabledisInternalTopicSeparatorEnabled ? separator + "internal" : ".internal"; } public String offsetSyncsTopic(String clusterAlias) // should use internalSeparator() to name the offset sync topic public String checkpointsTopic(String clusterAlias) // should use internalSeparator() to name the offset topic } |
Backporting plan
- The KIP need to be released as part of the bugfix release to the last 3 versions
- The KIP need to be backported into 3.1 and 3.2 (any other version reached EOL post KIP-690).
...
Users who upgrade from Kafka < 3.1 and wish to disable controlling internal topics using the separator will need to set `replication.policy.internal.topic.separator.enabled` to false. Any users upgrading from > 3.1 don't need to do anything.
Rejected Alternatives
- Introduce a LegacyReplicationPolicy that inherits
DefaultReplicationPolicy
that provide an override for ReplicationPolicy.offsetSyncsTopic and ReplicationPolicy.checkpointsTopic methods to use old topics and ignore the usage of the topic's separator. The advantage of this approach is avoiding introducing a new property to only fix a backward compatibility bug. However, this approach has a high risk of diversion between this new class andDefaultReplicationPolicy
at some point in the future. It also adds another class to maintain for MM2 without adding a big value for the connector besides fixing the backward compatibility bug.