...

1. Custom metrics are experimental.
2. Custom Metrics base on the New Metric System.
3. Allow to register essential ready-to-use metrics like LongMetric which can be exported by a providing metric exporter.
4. User can add own metrics into different registries.
5. Custom metrics are separated from the internals by registry name prefix "custom." (lower cased). 
6. Names of custom metrics and their registries can't contain any spaces.
7. Management of custom metrics might require a permission.
8. If we expose only few simple metrics, we should document examples of effecient metrics implementations like LongAdder.

Risks and Assumptions

• Custom metrics are just another way to hit performance and consume additional resources.
• There could be race conditions on metric registrations. Metrics with incompatible types can be concurrently registered with the same names.
• We do not provide read-only metric wrappers for the internal metrics. User can cast read-only metric to writeable implementation or interface and change value even of an iternal metric. But we already have public Metric#reset(), IgniteSpiContext#removeMetricRegistry(). And current ReadOnlyMetricRegistry doesn't return write-protected metric implementation.expose API of all internal metrics. At least with the first tickets.
• Custom metrics aren't stored and require re-registration after node restartWe do not expose API of all internal metrics. At least with the first tickets.
• Custom metrics aren't stored and require re-registration after node restart. At least with the first tickets.

API

1. APIs description

An API to expose internals read-only internal metrics was already proposed. Might be joined with the custom metrics.

API

APIs description

To give an user the ability to register additional metricsTo give an user the ability to register additional metrics, we could either:

• Refactor a bit our current metrics. Create interfaces for writable metric registry, metric manager. Create interfaces for metrics like int, double, long, longAdder, boolean, object and the gauge metrics (IntGauge, LongGauge, etc.).
• Provide tiny-enough facade with consumers and suppliers for int, long, double value metrics.

These two approaches are shown below.

...

Obtaining Custom Metrics

package org.apache.ignite;

...

    IgniteMetrics metrics();

}

...

Interfaces for existing metrics

Instead of #4, we could create interfaces for the existing metrics implementations.

...

IgniteMetric

package org.apache.ignite.metric;

/**
* Allows to manage custom metrics and to obtain read-only internal metrics.
*
* Note: Names of custom metric registries are required to start with 'custom.' (lower case) and may have additional
* dot-separated qualifiers. The prefix 'custon' is automatically added if missed. For example, if provided custom registry name
* is "a.b.c.mname", it is automatically extended to "custom.a.b.c.mname".
*
* Any custom name or dot-separated name part can't have any spaces and must not be empty. Spaces are removed.
*
* Examples of custom metric registry names: "custom", "custom.admin", "custom.admin.sessions", "custom.processes", etc.
*/
@IgniteExperimental
public interface IgniteMetrics extends Iterable<ReadOnlyMetricRegistry> {
    /**
    * Gets or creates custom metric registry named "custom." + {@code registryName}.
    *
    * @return {@link IgniteMetricRegistry} registry.
    */
    IgniteMetricRegistry customRegistry(String registryName);

    /**
    * Gets or creates custom metric registry named "custom.".
    *
    * @return {@link IgniteMetricRegistry} Certain read-only metric registry.
    */
    default IgniteMetricRegistry customRegistry() {
        return customRegistry(null@Nullable ReadOnlyMetricRegistry findRegistry(String registryName);
 
  }
    /**
    * Gets metric registry including the Ignite's internal registries.
    *
    * @return Certain read-only metric registry.
    Removes custom metric registry named "custom." + {@code registryName}. */

    @Nullable ReadOnlyMetricRegistry findRegistryvoid removeCustomRegistry(String registryName);
    /** */
    void removeCustomRegistry(String registryName);
    /** Removes custom metric registry with name '.custom'./
    default void removeCustomRegistry() {
        removeCustomRegistry(null);
    }
}

3.2 IgniteMetricRegistry

Probably should be named "MetricRegistry" with the renaming of internal "MetricRegistry" to "MetricRegistryImpl".  

}

IgniteMetricRegistry

Probably should be named "MetricRegistry" with the renaming of internal "MetricRegistry" to "MetricRegistryImpl".  

package org.apache.ignite.metric;

/**
* Metric registry. Allows to get, add or remove metrics.
*
* @see IgniteMetrics
* @see ReadOnlyMetricRegistry
*/
@IgniteExperimental
public interface IgniteMetricRegistry extends ReadOnlyMetricRegistry {
    /** @return New or previously registered {@link IntMetric}. */
    IntMetric register(String name, IntSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link LongMetric}. */
    LongMetric register(String name, LongSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link DoubleMetric}. */
    DoubleMetric register(String name, DoubleSupplier supplier, @Nullable String desc);

    /** @return New or previously registered {@link ObjectMetric}. */
    <T> ObjectMetric<T> register(String name, Supplier<T> supplier, Class<T> typepackage org.apache.ignite.metric;
/**
* Metric registry. Allows to get, add or remove metrics.
*
* @see IgniteMetrics
* @see ReadOnlyMetricRegistry
*/
@IgniteExperimental
public interface IgniteMetricRegistry extends ReadOnlyMetricRegistry {
    /** @return {@code True} if new metric was added. {@code False} is other metric already exists with the same name. */
    boolean gauge(String name, IntSupplier supplier, @Nullable String desc);

    /** @return {@code True} if new metric was added. {@code False} is other metric already exists with the same name. */
    boolean gaugeNew or previously registered {@link BooleanMetric}. */
    BooleanMetric register(String name, LongSupplier BooleanSupplier supplier, @Nullable String desc);

    /** @return New {@code True} if new metric was added. {@code False} is other metric already exists @link IntValueMetric} or previous one with the same name. */
    boolean gaugeIntValueMetric intMetric(String name, DoubleSupplier supplier, @Nullable String desc);

    /** @return New {@code True} if new metric was added. {@code False} is other metric already exists with @link LongValueMetric} or previous one with the same name.  */
    <T> boolean gaugeLongValueMetric longMetric(String name, Supplier<T> supplier, Class<T> type, @Nullable String desc);

    /** @return New {@code True} if new metric was added. {@code False} is other metric already exists @link LongValueMetric} or previous one with the same name. */
    boolean gaugeLongSumMetric longAdderMetric(String name, BooleanSupplier supplier, @Nullable String desc);

    /**
    * @return New {@link IntValueMetricDoubleValueMetric} or previous one with the same name. {@code Null} if previous metric exists and
    * is not a {@link IntValueMetric}.
    */
    @Nullable IntValueMetric intMetricDoubleValueMetric doubleMetric(String name, @Nullable String desc);

    /**
    * @return New {@link LongValueMetricObjectValueMetric} or previous one with the same name. {@code Null} if previous metric exists and
    * is not a {@link LongValueMetric}.
    */
    @Nullable LongValueMetric longMetric*/
    <T> ObjectValueMetric<T> objectMetric(String name, Class<T> type, @Nullable String desc);

    /**
    * @return New {@link LongValueMetric} or previous one Removes metrics with the same name. {@code Null} if previous metric exists and
    * is not a {@link LongValueMetric}.
    name}.*/
    @Nullable LongSumMetric longAdderMetricvoid remove(String name, @Nullable String desc);

    /**
    * @return New {@link DoubleValueMetric} or previous one with the same name. {@code Null} if previous metric exists and
    * is not a {@link DoubleValueMetric}.
    Resets all metrics of this metric registry. */
    @Nullable DoubleValueMetric doubleMetric(String name, @Nullable String descvoid reset();
    /**
    * @return New {@link CustomMetric} or previous one with the same name. {@code Null} if previous metric exists and
    * is not a {@link CustomMetric}.
    */
    @Nullable <T> AnyValueMetric<T> objectMetric(String name, Class<T> type, @Nullable String desc);
    /** Removes metrics with the {@code name}.*/
    void remove(String name);
    /** Resets all metrics of this metric registry. */
    void reset();
}

3.3 Updatable metric interfaces list

}

Updatable metric interfaces list

To the package "org.apache.ignite.metric" we add:

• BooleanValueMetric
• IntValueMetric
• LongValueMetric
• LongSumMetric (a long adder)
• DoubleValueMetric
• ObjectValueMetric<T> (an object metric)

Names like "LongMetric" or "ObjectMetric" we already have in To the package "org.apache.ignite.spi.metric" we add:

• BooleanValueMetric
• IntValueMetric
• LongValueMetric
• LongSumMetric (a long adder)
• DoubleValueMetric
• AnyValueMetric<T> (an object metric)

Names like "LongMetric" or "ObjectMetric" we already have in the package "org.apache.ignite.spi.metric".

3.4 Examples of updatable metrics

.

Examples of updatable metrics

package org.apache.ignite.metricpackage org.apache.ignite.metric;

/** Updatable object value metric. */
@IgniteExperimental
public interface AnyValueMetric<ObjectValueMetric<T> extends ObjectMetric<T> {
    /** Sets object metric value./
    void value(T value);
}

...

/** Updatable simple long value metric. */
@IgniteExperimental
public interface LongValueMetric extends LongSumMetric {
    /** Sets long metric value. */
    void value(long value);
}

...

Minimal custom interface

Instead of #3the interfeces set above, we could bring only a minimal metric management interface.package org.apache.ignite;
/**
* Allows to manage custom metrics and to obtain read-only internal metrics.
* <p>
* Metrics are grouped into registries (groups). Every metric has full name which is the conjunction of registry name
* and the metric short name. Within a registry metric has only its own short name.
* <p>
* Note: Names of custom metric registries are required to start with 'custom.' (lower case) and may have additional
* dot-separated qualifiers. The prefix is automatically added if missed. For example, if provided custom registry name
* is "a.b.c.mname", it is automatically extended to "custom.a.b.c.mname".
* <p>
* Any custom name or dot-separated name part cannot have spaces and must not be empty. Spaces are removed.
* <p>
* Examples of custom metric registry names: "custom", "custom.admin", "custom.admin.sessions", "custom.processes", etc.
*/
@IgniteExperimental
interface.

package org.apache.ignite;

/**
* Allows to manage custom metrics.
* <p>
* Metrics are grouped into registries (groups). Every metric has full name which is the conjunction of registry name
* and the metric short name. Within a registry metric has only its own short name.
* <p>
* Note: Names of custom metric registries are required to start with 'custom.' (lower case) and may have additional
* dot-separated qualifiers. The prefix is automatically added if missed. For example, if provided custom registry name
* is "a.b.c.mname", it is automatically extended to "custom.a.b.c.mname".
* <p>
* Any custom name or dot-separated name part cannot have spaces and must not be empty.
* <p>
* Examples of custom metric registry names: "custom.admin", "custom.admin.sessions", "custom.processes", etc.
*/
@IgniteExperimental
public interface IgniteMetrics extends Iterable<ReadOnlyMetricRegistry> {

    /*public interface IgniteMetrics extends Iterable<ReadOnlyMetricRegistry> {    /**
    * Adds a long value custom metric.
    *
    * @return New or previously registered long value metric. {@code Null} if previous metric is not a {@link LongConsumer}.
    */
    @Nullable LongConsumer longMetric(String registryName, String metricName, @Nullable String description);
    /**
    * Adds a double value custom metric.
    *
    * @return New or previously registered long value metric. {@code Null} if previous metric is not a {@link DoubleConsumer}.
   */
    @Nullable DoubleConsumer doubleMetricLongConsumer longMetric(String registryName, String metricName, @Nullable String description);

    /**
    * Adds a int value custom metric.
    *
    * @return New or previously registered long value metric. {@code Null} if previous metric is not an {@link IntConsumer}.
  double metric. */
    @Nullable IntConsumer booleanMetricDoubleConsumer doubleMetric(String registryName, String metricName, @Nullable String description);

    /**
    * Adds a long value custom metric.
    *
    * @return {@code True} if {@code supplier} has been registered as a new int metric. {@code False}, if a previous
    * value supplier already exists.
  @return New or previously registered int metric. */
    boolean longMetricIntConsumer booleanMetric(String registryName, String metricName, LongSupplier supplier, @Nullable String description);

    /**
    * Adds a double value custom metric.
    *
    * @return {@code True} if {@code supplier} has been registered as a new int metric. {@code False}, if a previous
    * value supplier already exists.
    long custom metric with the value supplier.*/
    boolean doubleMetricvoid longMetric(String registryName, String metricName, DoubleSupplier LongSupplier supplier, @Nullable String description);

    /**
    * Adds a int value double custom metric .
    *
    * @return {@code True} if {@code supplier} has been registered as a new int metric. {@code False}, if a previous
    * value supplier already exists.
    with the value supplier. */
     void doubleMetric(String registryName, String metricName, DoubleSupplier supplier, @Nullable String description);

    /** Adds a int custom metric with the value supplier. */
    boolean void intMetric(String registryName, String metricName, BooleanSupplier supplier, @Nullable String description);

    /** Removes certain custom metric. */
    void removeCustomMetric(String registryName, String metricName);

    /** Removes entire custom metric registry. */
    void removeCustomRegistry(String registryName);

    /** Provides custom or internal read-only metric registry. */
    @Nullable ReadOnlyMetricRegistry findRegistry(String registryName);
}

...

We already have implementations of more complex and useful metrics. We could also store custom metrics. Thus, the development stages might be:

1. An API to expose internals read-only internal metrics was already proposed. Might be joined with the custom metrics. By the methods like "findRegistry" we can return also read-only internal metric.
2. Extending the initial API with more complex metrics like Histogram or HitRate.
3. Introduce a permission of custom metric management.
4. Storing registered custom metrics.
5. Allowing to change settings of configurable custom metrics like histograms.

...