Page History

...

MirrorMaker2 uses a few internal topics to store some of its metadata. Records in these topics use binary formats and so they can't be directly dumped displayed but first need to be parsed and formatted. A way to easily parse and format records in these topics would be handy when investigating issues.

Kafka already has the concept of MessageFormatters used for other internal topics using binary format. We have OffsetsMessageFormatter, GroupMetadataMessageFormatter, TransactionLogMessageFormatter and the console consumer tool is able to use them to dump the content of __consumer_offsets for example.

The proposal is to provide similar Formatters for the MirrorMaker2 topics: checkpoints, heartbeats and offset-sync. At the same time, I propose publicly exposing a MessageFormatter class that replaces the internal MessageFormatter trait we currently have.
By That way the new Formatters can be in the mirror project.

Additionally, having a public interface , it will also enable users to build their own formatters that they could reuse with the console-consumer tool. For example, one could create a formatter that works with a schema registry, or a formatter that hides some fields based on the user identity.

...