Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Recovery metrics are first collected for each subtask and then sent back to the JobManager for the aggregation to a single number per Job (from all of the subtasks), as described in FLIP-384. From all of those aggregated metrics, a single recovery span trace is created on the JobManager. Thus this FLIP must define not only API to report a custom attribute, but also how to aggregate it later.Custom metrics will be aggregated the same way how the built-in metrics are aggregated in the FLIP-384 (using max and sum). 

Public Interfaces

It will be possible to set the custom attributes via the following interfaces:

Code Block
@Experimental
public interface CustomMetric {
    enum AggregationMethod {
        MIN,
        MAX,
        SUM,
        AVG
    }

    CustomMetric addAggregationMethod(AggregationMethod aggregationMethod);
}

@Experimental
public interface CustomInitializationMetrics {
    CustomMetricvoid addMetric(String name, long value);
    CustomMetricvoid addMetric(String name, double value);
}

// intendend usage:
customInitializationMetrics.addMetric("DownloadTime", value)
    .addAggregationMethod(AggregationMethod.MAX)
    .addAggregationMethod(AggregationMethod.SUM)

The remaining issue is to pass CustomInitializationMetrics instance to for example RocksDBIncrementalRestoreOperation . In this FLIP I’m proposing to do it via adding the following interfaces to the org.apache.flink.runtime.state.StateBackend interface:

...