Metrics Collector API Specification

Sending Metrics to AMS (POST)

Sending metrics to Ambari Metrics Service can be achieved through the following API call.

The Sink implementations responsible for sending metrics to AMS, buffer data for 1 minute before sending. TimelineMetricCache provides a simple cache implementation to achieve this behavior.

Sample sink implementation use by Hadoop daemons: https://github.com/apache/ambari/tree/trunk/ambari-metrics/ambari-metrics-hadoop-sink

POST http://<ambari-metrics-collector>:6188/ws/v1/timeline/metrics

{
  "metrics": [
    {
      "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",
      "appid": "amssmoketestfake",
      "hostname": "ambari20-5.c.pramod-thangali.internal",
      "timestamp": 1432075898000,
      "starttime": 1432075898000,
      "metrics": {
        "1432075898000": 0.963781711428,
        "1432075899000": 1432075898000
      }
    }
  ]
}

Connecting (POST) to <ambari-metrics-collector>:6188/ws/v1/timeline/metrics/
Http response: 200 OK

Fetching Metrics from AMS (GET)

Sample call

GET http://<ambari-metrics-collector>:6188/ws/v1/timeline/metrics?metricNames=AMBARI_METRICS.SmokeTest.FakeMetric&appId=amssmoketestfake&hostname=<hostname>&precision=seconds&startTime=1432075838000&endTime=1432075959000

Http response: 200 OK

Http data:

{

   "metrics": [

      {

         "timestamp": 1432075898089,

         "metricname": "AMBARI_METRICS.SmokeTest.FakeMetric",

         "appid": "amssmoketestfake",

         "hostname": "ambari20-5.c.pramod-thangali.internal",

         "starttime": 1432075898000,

         "metrics": {

            "1432075898000": 0.963781711428,

            "1432075899000": 1432075898000

         }

      }

   ]

}

Generic GET call format

http://<AMS_HOST>:6188/ws/v1/timeline/metrics?metricNames=<>&hostname=<>&appId=<>&startTime=<>&endTime=<>&precision=<>

Query Parameters Explanation

AMS Metadata API

 AMS has 2 metadata endpoints that are useful for finding out the set of metrics it received, as well as the topology of the cluster.

METRICS METADATA

Endpoint : http://<AMS_HOST>:6188/ws/v1/timeline/metrics/metadata

Data returned : A mapping between the set of APP_IDs to the list of metrics received with that AppId. 

Sample data returned

HOSTS METADATA

Endpoint : http://<AMS_HOST>:6188/ws/v1/timeline/metrics/hosts

Data returned : A mapping between the hosts in the cluster and the set of APP_IDs on the host. 

Sample data returned

Guide to writing your own Sink

• Include the ambari-metrics-common artifacts from source or maven-central (when available) into your project
• Find below helpful info regarding common data-structures to use from the ambari-metrics-common module
• Extend the org.apache.hadoop.metrics2.sink.timeline.AbstractTimelineMetricsSink class and implement the required methods
• Use the org.apache.hadoop.metrics2.sink.timeline.cache.TimelineMetricsCache to store intermediate data until it is time to send (example: collection interval = 10 seconds, send interval = 1 minute). The cache implementation provides the logic needed for buffering and local aggregation.
• Use org.apache.hadoop.metrics2.sink.timeline.AbstractTimelineMetricsSink#emitMetrics to send metrics to AMS backend.

METRIC DATA STRUCTURE

Source location for common data structures module: https://github.com/apache/ambari/tree/trunk/ambari-metrics/ambari-metrics-common/

Example sink implementation: https://github.com/apache/ambari/blob/trunk/ambari-metrics/ambari-metrics-hadoop-sink/

INTERNAL PHOENIX KEY STRUCTURE

 The Metric Record Key data structure is described below: