Motivation

It is useful for user to work with own metrics, not only with provided by Ignite. Current public metrics API doesn't expose any method to add or delete additional metrics.

The most important reason to provide custom is probably the convenience of collecting of desired metrics using one platform, the same client, through the same API. This feature can simplify user application.

Examples of custom metric usages

• Some metrics may not be provided yet and user registers own.
• Metrics for load balancing.
• Various metrics of business processes or related to user application.
• Administration metrics.
• Security-related metrics.

Examples of databases with custom metrics

Oracle DB , Postgres , Oracle Coherence , MS SQL Server , IBM DB2

Implementation proposal

General approach

• Custom Metrics are based on the New Metric System.
• Allow to register essential ready-to-use metrics like LongMetric which can be exported by a existing metric exporter.
• User can register own metrics into different registries.
• Custom metrics are separated from the internals by registry name prefix "custom.".
• Naming strategy of metrics and their registries are the same as of the provided metrics.
• Management of custom metrics might require a permission.
• We should document examples of effecient metrics implementations like LongAdder.

API

   1. New metrics facade to Ignite:

In future, we might want to provide an internal API to obtain metrics like this. It could be quite different and we should split them. 

package org.apache.ignite;

public interface Ignite {

   public IgniteCustomMetrics metrics();

}

   2. New interface IgniteCustomMetrics

To keep custom metrics detached from the internals, we should either automaticall add a name prefix or throw an exception if provided name doesn't start with that prefix.

package org.apache.ignite;

/**
 * Manages custom metrics. 
 *
 * Metrics are grouped into registries. Every metric has full name which is conjunction of registry name and metric name. 
 * Registry name is part of the full name until last '.'. Metric name is the last part of the full name. 
 * 
 * Full names and registry names are always prefixed with "custom.". For example, if provided full name is "a.b.c.mname", 
 * it is automatically converted to "custom.a.b.c.mname". If full name is "custom.a.b.c.mname", it is used as is. 
 * Where "custom.a.b.c" is a registry name and "mname" is a metric name. 
 */
public interface IgniteCustomMetrics {
    /**
     * Registers metric. Adds new metric registry if absent. 
     *
     * @return Previous metric if it already exists. Otherwise, {@code metric}. 
     */
    <T extends Metric> T metric(String fullName, T metric);

    /** Removes metric. */
    void removeMetric(String fullName);

    /** @return Certain metric registry. */
    ReadOnlyMetricRegistry registry(String registryName);

    /** Removes certain registry and all its metrics. */
    void removeRegistry(String registryName);


    // Essential metrics. Int, Long and Double should be enough.

    /**
     * Registers a {@link LongMetric}. Adds new metric registry if absent.
     *
     * @return Previous long value setter if it already exists. A new one otherwise.
     */
    LongConsumer longMetric(String fullName, @Nullable String description);

    /**
     * Registers a {@link DoubleMetric}. Adds new metric registry if absent.
     *
     * @return Previous double value setter if it already exists. A new one otherwise.
     */
    DoubleConsumer doubleMetric(String fullName, @Nullable String description);

    /**
     * Registers a {@link IntMetric}. Adds new metric registry if absent.
     *
     * @return Previous int value setter if it already exists. A new one otherwise.
     */
    IntConsumer booleanMetric(String fullName, @Nullable String description);



    // Might be useful the gauges.

    /**
     * Registers a {@link LongMetric} metric with value supplier {@code valueSupplier}. Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if meric already exists.  
     */
    boolean longMetric(String fullName, LongSupplier valueSupplier, @Nullable String description);

    /**
     * Registers a {@link DoubleMetric} metric with value supplier {@code valueSupplier}. Adds new metric registry if absent.
     *
     * @return {@code True} if metric was added. {@code False} if meric already exists.  
     */
    boolean doubleMetric(String fullName, DoubleSupplier valueSupplier, @Nullable String description);

    /**
     * Registers a {@link IntMetric} metric with value supplier {@code valueSupplier}. Adds new metric registry if absent.
     * 
     * @return {@code True} if metric was added. {@code False} if meric already exists.  
     */
    boolean intMetric(String fullName, BooleanSupplier valueSupplier, @Nullable String description);
}

Risks and Assumptions

• Custom metrics can affect the performance.
• There could be race conditions on metric registrations.
• We do not provide varuous useful metric implementatons.
• Custom metrics aren't stored and require re-registration after node restart. At least at the first phase.

Further Steps

We already have implementations of more complex and useful metrics. We could also store custom metrics. Thus, the development stages might be:

1. Implementation of the initial API.
2. Extending this API with metrics like Histogram, HitRate, LongAdder.
3. Introduce a permission of custom metric management.
4. Storing registered custom metrics.
5. Allowing to change settings of configurable custom metrics like histograms.

References

1. IEP-35 Monitoring & Profiling
2. New Metric System
3. Ticket of a public metric API
4. IEP-116 : Ignite 3 metric

Discussion Links

Tickets