...

Motivation

Currently Paimon runs without monitoring and metrics out of box. Users In the production environment, users need to know how their Paimon Table behaves like what is the commit duration, how many files each commit generatedadded or deleted, the status of compaction operation, the performance duration of scan query, lag of streaming reading and so on. So we need to introduce a metrics system and more metrics for Paimon.  

...

public interface MetricsReporter {
	/** Configure reporter after instantiating it.*/
     void open();

    /** Closes this reporter. */
    void close();

	/** Report the current measurements. This method is called periodically by the Metrics. */
	void report();
}

...

Proposed Changes

Architecture

Metrics Registering

Take CommitMetrics as example, the CommitMetrics will be instantiated by FileStoreCommitImpl, then commit related metrics will be registered by MetricRegistry in singleton Metrics. 

The Metrics has instance of MetricRegistry and MetricsReporters set. MetricRegistry maintains metrics map containers. Metrics registering is a process of putting metrics instances into the metric (gauge, counter) map container. 

Update metrics value

The CommitMetrics values will be updated around commit() operation, for example the commit starting time will be recorded before commit operation, CommitDuration value will be recorded after commit completing.

CompactionMetrics values will be updated around compaction operation, and ScanMetrics will be recorded through the scan planning process.

Report metrics to external backend

Each reporter instance has an timer task fetching metrics from the metrics containers periodically and report them out to the external backends. Paimon will has a default reporter backend with JMX, users can define their own MetricsReporter by implement MetricsReporter  interface. Here we can introduce a core option metrics.reporter  to specify a metrics backend.

Metrics list

We introduce CommitMetrics , CompactionMetrics , ScanMetrics as metrics set to measure the stats of Paimon table committing, compaction and scanning.

CommitMetrics

public class CommitMetrics {
	private Metrics metrics;
	private final String COMMIT_DURATION_METRIC = "commitDuration";
	...
	private void registerCommitMetrics(Metrics metrics) {
		metrics.gauge(COMMIT_DURATION_METRIC, new CommitDurationTimer());
		...
	}
	...
}

...

public class CompactionMetrics {
	private Metrics metrics;
	private final String COMPACTED_FILES_METRIC = "compactedFiles";
	...
	private void registerCompactionMetrics(Metrics metrics) {
		metrics.counter(COMPACTED_FILES_METRIC, new CompactedFilesCounter());
		...
	}
	...
}

Proposed Changes

Architecture

...

Metrics Registering

Take CommitMetrics as example, the CommitMetrics will be instantiated by TableCommitImpl, then commit related metrics will be registered by MetricRegistry in singleton Metrics. 

The Metrics has properties of MetricRegistry and MetricsReporters set. MetricRegistry maintains metrics map containers. Metrics registering is a process of putting metrics instances into the metric (gauge, counter) map container. 

Update metrics value

The CommitMetrics values will be updated around commit() operation, for example the commit starting time will be recorded before commit operation, CommitDuration value will be recorded after commit completing.

CompactionMetrics values will be updated around compaction operation, and ScanMetrics will be recorded through the scan planning process.

Report metrics to external backend

Each reporter instance has an timer task fetch metrics from the metrics containers periodically and report them out to the external backends. Paimon will has a default reporter backend with JMX, users can define their own MetricsReporter by implement `MetricsReporter` interface.

...

Commit Metrics

Scan Metrics

...