Motivation

It is useful for user to work with own metrics, not only with provided by Ignite. Current public metrics API doesn't expose any method to add or delete additional metrics.

The most important reason to provide custom is probably the convenience of collecting of desired metrics using one platform, the same client, through the same API. This feature can simplify user application.

Famous databases with custom metrics: Oracle DB , Postgres , Oracle Coherence , MS SQL Server , IBM DB2

Examples of custom metric usages:

• Some metrics may not be provided yet and user registers own.
• Metrics for load balancing.
• Custom conflict resolver successes/fails.
• Various metrics of business processes or related to user application.
• Administration metrics.
• Security-related metrics.

Implementation proposal

1. Custom Metrics base on the New Metric System.
2. Allow to register essential ready-to-use metrics like LongMetric which can be exported by a providing metric exporter.
3. User can add own metrics into different registries.
4. Custom metrics are separated from the internals by registry name prefix "custom." (lower cased). 
5. Names of custom metrics and their registries can't contain any spaces.
6. Management of custom metrics might require a permission.

Risks and Assumptions

• Custom metrics are just another way to hit performance and consume additional resources.
• There could be race conditions on metric registrations. Metrics with incompatible types can be concurrently registered with the same names.
• We do not expose API of all internal metrics. At least with the first tickets.
• Custom metrics aren't stored and require re-registration after node restart. At least with the first tickets.

API

APIs description

To give an user the ability to register additional metrics, we could either:

• Refactor a bit our current metrics. Create interfaces for writable metric registry, metric manager. Create interfaces for metrics like int, double, long, longAdder, boolean, object and the gauge metrics (IntGauge, LongGauge, etc.).
• Provide tiny-enough facade with consumers and suppliers for int, long, double value metrics.

These two approaches are shown below.

Obtaining Custom Metrics

package org.apache.ignite;

public interface Ignite {

    IgniteMetrics metrics();

}

Proposed APIs: Interfaces for existing metrics

IgniteMetric

package org.apache.ignite.metric;

/**
* Allows to manage custom metrics.
*
* Note: Names of custom metric registries are required to start with 'custom.' (lower case) and may have additional
* dot-separated qualifiers. The prefix 'custom' is automatically added if missed. For example, if provided custom registry name
* is "a.b.c.mname", it is automatically extended to "custom.a.b.c.mname".
*
* Any custom name or dot-separated name part can't have any spaces and must not be empty.
*
* Examples of custom metric registry names: "custom.admin", "custom.admin.sessions", "custom.processes", etc.
*/
@IgniteExperimental
public interface IgniteMetrics extends Iterable<ReadOnlyMetricRegistry> {
    /**
    * Gets or creates custom metric registry named "custom." + {@code registryName}.
    *
    * @return {@link IgniteMetricRegistry} registry.
    */
    IgniteMetricRegistry customRegistry(String registryName);

    /**
    * Gets custom metric registry.
    *
    * @return Certain read-only metric registry.
    */
    @Nullable ReadOnlyMetricRegistry findRegistry(String registryName);

    /** Removes custom metric registry named "custom." + {@code registryName}. */

    void removeCustomRegistry(String registryName);
}

IgniteMetricRegistry

Probably should be named "MetricRegistry" with the renaming of internal "MetricRegistry" to "MetricRegistryImpl".  

package org.apache.ignite.metric;

/**
* Metric registry. Allows to get, add or remove metrics.
*
* @see IgniteMetrics
* @see ReadOnlyMetricRegistry
*/
@IgniteExperimental
public interface IgniteMetricRegistry extends ReadOnlyMetricRegistry {
    /** @return New or previously registered {@link IntMetric}. */
    IntMetric register(String name, IntSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link LongMetric}. */
    LongMetric register(String name, LongSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link DoubleMetric}. */
    DoubleMetric register(String name, DoubleSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link ObjectMetric}. */
    <T> ObjectMetric<T> register(String name, Supplier<T> supplier, Class<T> type, @Nullable String desc);

    /** @return New or previously registered {@link BooleanMetric}. */
    BooleanMetric register(String name, BooleanSupplier supplier, @Nullable String desc);

    /** @return New {@link IntValueMetric} or previous one with the same name. */
    IntValueMetric intMetric(String name, @Nullable String desc);

    /** @return New {@link LongValueMetric} or previous one with the same name.  */
    LongValueMetric longMetric(String name, @Nullable String desc);

    /** @return New {@link LongValueMetric} or previous one with the same name. */
    LongSumMetric longAdderMetric(String name, @Nullable String desc);

    /** @return New {@link DoubleValueMetric} or previous one with the same name.  */
    DoubleValueMetric doubleMetric(String name, @Nullable String desc);

    /** @return New {@link ObjectValueMetric} or previous one with the same name. */
    <T> ObjectValueMetric<T> objectMetric(String name, Class<T> type, @Nullable String desc);

    /** Removes metrics with the {@code name}.*/
    void remove(String name);

    /** Resets all metrics of this metric registry. */
    void reset();
}

Updatable metric interfaces list

To the package "org.apache.ignite.metric" we add:

• BooleanValueMetric
• IntValueMetric
• LongValueMetric
• LongSumMetric (a long adder)
• DoubleValueMetric
• ObjectValueMetric<T> (an object metric)

Names like "LongMetric" or "ObjectMetric" we already have in the package "org.apache.ignite.spi.metric".

Examples of updatable metrics

package org.apache.ignite.metric;

/** Updatable object value metric. */
@IgniteExperimental
public interface ObjectValueMetric<T> extends ObjectMetric<T> {
    /** Sets object metric value./
    void value(T value);
}

/** Updatable simple double value metric. */
@IgniteExperimental
public interface DoubleValueMetric extends DoubleMetric {
    /** Raises metric value. */
    void add(double value);

    /** Sets double metric value. */
    void value(double value);
}

/** Updatable long metric which is efficient with adding values. Calculates sum in {@link LongMetric#value()}. */
@IgniteExperimental
public interface LongSumMetric extends LongMetric {
    /** Raises metric value. */
    void add(long value);

    /** Increments metric value with 1L. */
    void increment();

    /** Decrements metric value with -1L. */
    void decrement();
}

/** Updatable simple long value metric. */
@IgniteExperimental
public interface LongValueMetric extends LongSumMetric {
    /** Sets long metric value. */
    void value(long value);
}

API alternative: single minimal facade

Instead of the interfeces set above, we could bring only a minimal metric management interface.

package org.apache.ignite;

/**
* Allows to manage custom metrics.
* <p>
* Metrics are grouped into registries (groups). Every metric has full name which is the conjunction of registry name
* and the metric short name. Within a registry metric has only its own short name.
* <p>
* Note: Names of custom metric registries are required to start with 'custom.' (lower case) and may have additional
* dot-separated qualifiers. The prefix is automatically added if missed. For example, if provided custom registry name
* is "a.b.c.mname", it is automatically extended to "custom.a.b.c.mname".
* <p>
* Any custom name or dot-separated name part cannot have spaces and must not be empty.
* <p>
* Examples of custom metric registry names: "custom.admin", "custom.admin.sessions", "custom.processes", etc.
*/
@IgniteExperimental
public interface IgniteMetrics extends Iterable<ReadOnlyMetricRegistry> {

    /** @return New or previously registered long metric. */
    LongConsumer longMetric(String registryName, String metricName, @Nullable String description);

    /** @return New or previously registered double metric. */
    DoubleConsumer doubleMetric(String registryName, String metricName, @Nullable String description);

    /** @return New or previously registered int metric. */
    IntConsumer booleanMetric(String registryName, String metricName, @Nullable String description);

    /** Adds a long custom metric with the value supplier.*/
    void longMetric(String registryName, String metricName, LongSupplier supplier, @Nullable String description);

    /** Adds a double custom metric with the value supplier. */
     void doubleMetric(String registryName, String metricName, DoubleSupplier supplier, @Nullable String description);

    /** Adds a int custom metric with the value supplier. */
    void intMetric(String registryName, String metricName, BooleanSupplier supplier, @Nullable String description);

    /** Removes certain custom metric. */
    void removeCustomMetric(String registryName, String metricName);

    /** Removes entire custom metric registry. */
    void removeCustomRegistry(String registryName);

    /** Provides custom read-only metric registry. */
    @Nullable ReadOnlyMetricRegistry findRegistry(String registryName);
}

Code examples

Further Steps

We already have implementations of more complex and useful metrics. We could also store custom metrics. Thus, the development stages might be:

1. An API to expose internals read-only internal metrics was already suggested. Might be joined with the custom metrics. By the methods like "findRegistry" we can return also read-only internal metrics.
2. Extending the initial API with more complex metrics like Histogram or HitRate.
3. Introduce a permission for custom metric management.
4. Storing registered custom metrics.
5. Allowing to change settings of configurable custom metrics like histograms.

References

1. IEP-35 Monitoring & Profiling
2. New Metric System
3. Ticket of a public metric API
4. IEP-116 : Ignite 3 metric

Discussion Links

Tickets