...

1. The user running MM2 has the following ACLs
   • `Create` ACLs for topics on the source clusters to create heartbeat topics.
   • `Create`  and `Alter/AlterConfigs` ACLs to create topics, create partitions, update topics' config and topics' ACLs on the destination clusters.
2. MM2 can bypass any existing Kafka resource management solutions that organizations runs and create/update resources outside this system.

These assumption wouldn't work for any organization that runs any sort of resource management or federated solutions where these systems are usually the only application allowed to initializing a client with `Create` and`Alter/AlterConfigs` ACLs. And no other teams/groups/applications are allowed to have this same level of ACLs to create such a client outside these systems.

...

As mentioned before the usage of AdminClient directly within MM2 simplify the resource management for MM2. However, it does create the following problems for any users who use IaC (Infra-as-Code), federated solutions, or have a capacity/budget planning system for Kafka destination clusters. Here is a list of potential undesired impact of MM2 bypass the organization ecosystem:

...

This KIP proposes a way for users to run MM2 with custom implementation for the Kafka resource manager in order to easily integrate MM2 with their ecosystem. 

Public Interfaces

The KIP proposes adding add flexibility to how MM2 manage Kafka resources. 

There're 2 potential solutions under discussion now: 

1. Adding a new interface called KafkaResourceManager that defines how MM2 will create, modify, and list any Kafka topics and ACLs. MirrorMaker2’s original behaviour will be kept in DefaultResourceManager.

   The implemented class can be overridden using the following configurations.

   • resource.manager.class default value will be set to DefaultResourceManager
   • or can be configured based on cluster aliases using <cluster_alias>.resource.manager.class
   The downside with this is it may create confusing for users between Admin and ResourceManager 
2. Allow MM2 to load different implementation of Admin interface

The implemented class can be overridden using the following configurations.

• • admin.class default value will be set to AdminClient
  • or can be configured based on cluster aliases using <cluster_alias>.admin.class

The downsides of this proposal are

1. 1. 1.  The Admin interface has 86 methods, and MM2 depends on 13 of them; this means any custom implementation of the Admin interface will be humongous. The user will need to return "NotImplementedException" for 73 methods that MM2 does not use. 
      2. MM2 interactions with AdminClient is not listed in one place, so it's bit hard to list what interfaces you need to implement to MM2. 
      3. Keeping up with the AdminClient interface which is enormous. 

Proposed Changes

There're 2 potential solutions under discussion now: 

1. Add new interface for MM2 to manage resources KafkaResourceManager 

KafkaResourceManager interface will define how to list, create, update topics, topic configs, and ACLs.

...

DefaultResourceManager will use AdminClient to manage the resources. And default value for resource.manager.class will be DefaultResourceManager for backward compatibility.

2. Add configuration to MM2 to load different implementation of Admin interface

1. MM2 will use implementation form admin.class  to initialize admin client.
2. MM2 interactions with Admin interface need to be in one place to make it easier to list what interfaces need to be implemented when users provide their own Admin implementation 

Compatibility, Deprecation, and Migration Plan

• When users upgrade an existing MM2 cluster they don’t need to change any of their current configuration as this proposal maintains the default behaviour for MM2.

Rejected Alternatives

1. Manage creating and modifying Kafka topics and ACLs outside MM2 by building a separate tool that monitors the same set of topics as MirrorMaker2 and create/modify topics and ACLs once it detects configuration changes. The downsides with this are
   
   1. This will be a duplicate effort as MM2 already has all this logic implemented; it only needs to use a different client than AdminClient.
      
   2. MM2 still bypass any capacity safeguards the ecosystem would have in place. 
   3. Any downtime with this tool will cause conflict between MM2 mirrored resources and management resource system
.Allow MM2 users to provide a custom implementation of the Admin interface instead of adding a new interface. The downsides with this are
1.  The Admin interface has 86 methods, and MM2 depends on 13 of them; this means any custom implementation of the Admin interface will be humongous. The user will need to return "NotImplementedException" for 73 methods that MM2 does not use. 
2. MM2 interactions with AdminClient is not listed in one place, so it's bit hard to list what interfaces you need to implement to MM2. 
Keeping up with the AdminClient interface which is enormous
1. 1. .