...
Recovery metrics are first collected for each subtask and then sent back to the JobManager for the aggregation to a single number per Job (from all of the subtasks), as described in FLIP-384. From all of those aggregated metrics, a single recovery span trace is created on the JobManager. Thus this FLIP must define not only API to report a custom attribute, but also how to aggregate it later.Custom metrics will be aggregated the same way how the built-in metrics are aggregated in the FLIP-384 (using max and sum).
Public Interfaces
It will be possible to set the custom attributes via the following interfaces:
Code Block |
---|
@Experimental public interface CustomMetric { enum AggregationMethod { MIN, MAX, SUM, AVG } CustomMetric addAggregationMethod(AggregationMethod aggregationMethod); } @Experimental public interface CustomInitializationMetrics { CustomMetricvoid addMetric(String name, long value); CustomMetricvoid addMetric(String name, double value); } // intendend usage: customInitializationMetrics.addMetric("DownloadTime", value) .addAggregationMethod(AggregationMethod.MAX) .addAggregationMethod(AggregationMethod.SUM) |
The remaining issue is to pass CustomInitializationMetrics instance to for example RocksDBIncrementalRestoreOperation . In this FLIP I’m proposing to do it via adding the following interfaces to the org.apache.flink.runtime.state.StateBackend interface:
...