Motivation

It is useful for user to work with own metrics, not only with provided by Ignite. Current public metrics API doesn't expose any method to add or delete additional metrics.

The most important reason to provide custom is probably the convenience of collecting of desired metrics using one platform, the same client, through the same API. This feature can simplify user application.

Examples of custom metric usages

• Some metrics may not be provided yet and user registers own.
• Metrics for load balancing.
• Various metrics of business processes or related to user application.
• Administration metrics.
• Security-related metrics.

Examples of databases with custom metrics

Oracle DB , Postgres , Oracle Coherence , MS SQL Server , IBM DB2

Implementation proposal

General approach

• Custom Metrics are based on the New Metric System.
• Provide essential ready-to-use metrics like LongMetric which can be exported by a existing metric exporter.
• User can store own metrics in different registers corresponding to metrics groups.
• Custom metrics are separated from the internals by registry name prefix "custom.".
• Custom metrics management might require a permission.
• We should document examples of effecient metrics implementations like LongAdder.

API

   1. New metrics facade to Ignite:

In future, we might want to provide an internal API to obtain metrics like this. It could be quite different and we should split them. 

package org.apache.ignite;

public interface Ignite {

   public IgniteCustomMetrics metrics();

}

   2. New interface IgniteCustomMetrics

To keep custom metrics detached from the internals, we should either automaticall add a prefix to custom name or throw an exception if provided registry name doesn't start with that prefix.

package org.apache.ignite;

/**
 * Manages custom metrics. Names of custom metrics registers are always stored with the prefix "custom.".
 * If provided registry name is "a.b.regname", it is automatically converted to "custom.a.b.regname".
 * If the name is "custom.a.b.regname", it is used as is.  
 */
public interface IgniteCustomMetrics {
    /**
     * Registers metric in registry {@code registryName}. Adds new metric registry if absent. 
     *
     * @return Previous metric if it already exists with the same name. {@code metric} otherwise. 
     */
    <T extends Metric> T metric(String registryName, String metricName, T metric);

    /** Removes metric from registry {@code registryName}. */
    void removeMetric(String registryName, String metricName);

    /** @return Metric registry named '{@code registryName}'. */
    ReadOnlyMetricRegistry registry(String registryName);

    /** Removes registry named '{@code registryName}'. */
    void removeRegistry(String registryName);


    // Essential metrics. Int, Long and Double should be enough.

    /**
     * Registers a {@link LongMetric} in registry {@code registryName}. Adds new metric registry if absent.
     *
     * @return Previous long value setter if it already exists with the same name. A new one otherwise.
     */
    LongConsumer longMetric(String registryName, String metricName, @Nullable String description);

    /**
     * Registers a {@link DoubleMetric} in registry {@code registryName}. Adds new metric registry if absent.
     *
     * @return Previous double value setter if it already exists with the same name. A new one otherwise.
     */
    DoubleConsumer doubleMetric(String registryName, String metricName, @Nullable String description);

    /**
     * Registers a {@link IntMetric} in registry {@code registryName}. Adds new metric registry if absent.
     *
     * @return Previous int value setter if it already exists with the same name. A new one otherwise.
     */
    IntConsumer booleanMetric(String registryName, String metricName, @Nullable String description);



    // Might be useful the gauges.

    /**
     * Registers a {@link LongMetric} metric in registry {@code registryName} with long value supplier {@code valueSupplier}. 
     * Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if another meric already exists under the same name.  
     */
    boolean longMetric(String registryName, String metricName, LongSupplier valueSupplier, @Nullable String description);

    /**
     * Registers a {@link DoubleMetric} metric in registry {@code registryName} with double value supplier {@code valueSupplier}. 
     * Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if another meric already exists under the same name.  
     */
    boolean doubleMetric(String registryName, String metricName, DoubleSupplier valueSupplier, @Nullable String description);

    /**
     * Registers a {@link IntMetric} metric in registry {@code registryName} with int value supplier {@code valueSupplier}. 
     * Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if another meric already exists under the same name.  
     */
    boolean intMetric(String registryName, String metricName, BooleanSupplier valueSupplier, @Nullable String description);


    // Additionnaly, we might expose an object metrics API.

    /**
     * Registers a {@link ObjectMetric} metric in registry {@code registryName} with object value supplier {@code valueSupplier}. 
     * Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if another meric already exists under the same name.  
     */
    <T> boolean objectMetric(String registryName, String metricName, Class<T> valueType, Supplier<T> valueSupplier, @Nullable String description);

    /**
    * Registers a {@link ObjectMetric} in registry {@code registryName} with object value supplier {@code valueSupplier}. 
    * Adds new metric registry if absent.
    *
    * @return New or previously registered object setter.
    */
    <T> Consumer<T> objectMetric(String registryName, String metricName, Class<T> valueType, Supplier<T> valueSupplier, @Nullable String description);
}

Risks and Assumptions

• Custom metrics can affect the performance.
• There could be race conditions on metric registrations.
• We do not provide varuous useful metric implementatons.
• Custom metrics aren't stored and require re-registration after node restart. At least at the first phase.

Further Steps

We already have implementations of more complex and useful metrics. We could also store custom metrics. Thus, the development stages might be:

1. Implementation of the initial API.
2. Extending this API with metrics like Histogram, HitRate, LongAdder.
3. Introduce a permission of custom metric management.
4. Storing registered custom metrics.
5. Allowing to change settings of configurable custom metrics like histograms.

References

1. IEP-35 Monitoring & Profiling
2. Ticket of a public metric API
3. IEP-116 : Ignite 3 metric

Discussion Links

Tickets