Status

Current state: Draft

Discussion thread: TBD

JIRA: KAFKA-14740

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

MirrorSourceConnector registers a few metrics to track the mirroring of records from the source to the target cluster. Its metrics have the "target", "topic", "partition" tags. By default, with DefaultReplicationPolicy, the topic name contains the source cluster alias as its prefix, for example source.topic. However as it's part of the topic tag it's oftentimes not easily usable in monitoring tools. In addition, with IdentityReplicationPolicy the source cluster alias does not appear in the metrics at all.

With multiple mirroring routes, it can be helpful to aggregate metrics based on a specific source or target cluster to better gauge the usage and state of mirroring deployments.

Public Interfaces

A new tag, source, will be added to all metrics registered by MirrorSourceConnector. It will contain the source cluster alias. For example:
kafka.connect.mirror:type=MirrorSourceConnector,source=one,target=two,topic=newtopic,partition=0

The new tag will be added in the front of the list of tags so the tag order is the same as MirrorCheckpointConnector metrics.

Proposed Changes

Update MirrorSourceMetrics to have a new tag, source, and set it to the source cluster alias.

Compatibility, Deprecation, and Migration Plan

This may break queries in monitoring tools that serialize tags in metric names.

Test Plan

This will be tested using unit tests.

Rejected Alternatives

• Add the source tag at the end of the list of tags: Since this has similar compatibility implications than the proposed method, I favored having the same tag order in MirrorSourceConnector and MirrorCheckpointConnector.