Current state: Under Discussion
Discussion thread: here
JIRA: KAFKA-13365
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
As of present, MirrorMaker 2 (aka MM2) 's client configurtaion feature has some problems:
bootstrap.servers
, security.protocol
, ssl, sasl, etc; that is, a configuration like 'A→B.producer.batch.size'
is ignored.MirrorSourceConnector
, MirrorCheckpointConnector
, and MirrorHeartbeatConnector
) are source connectors, they use producer instance created by Kafka Connector, which uses 'producer.override.{property-name}'
in connector configuration; But, 'target.producer.{property-name}'
are not automatically applied to 'producer.override.{property-name}'
so not actually applied to producer instance.'bootstrap.servers'
of the clusters in cluster-level, like 'A.bootstrap.servers'
or 'B.bootstrap.servers'
; but it also allows to override them in cluster-level and replication-level configs, like 'A.producer.bootstrap.servers'
or 'A→B.consumer.bootstrap.servers'
; actually these configurations are not used but, it would be better to ignore it and give a warning.For the proof of the problem, please refer here.
This proposal does not introduce any new public interface or configurable settings; it only makes the existing ones (which are currently ignored by MirrorMaker 2) working by providing consistent, easy-to-understand configuration rules.
A->B.producer.{property-name}
and A->B.admin.{property-name}
are applied to target cluster clients and A->B.consumer.{property-name}
is applied to source cluster clients.target.producer.{property-name}
is automatically copied to producer.override.{property-name}
. In connector mode, the user should manually configure them, and the documentation will explicitly mention it.Since this proposal only makes currently not working settings work, there is no deprecation or migration plan.
None.