Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
(Work in progress)

 

Ambari Metrics System ("AMS") is a system for collecting, aggregating and serving Hadoop and system metrics in Ambari-managed clusters.

Info

The JIRA Epic for Ambari Metrics System delivery is https://issues.apache.org/jira/browse/AMBARI-5707 and was first introduced with Ambari 2.0.0.

Terminology

TermDefinition

Ambari Metrics System (“AMS”)

The built-in metrics collection system for Ambari.

Metrics Collector

The standalone server that collects metrics, aggregates metrics, serves metrics from the Hadoop service sinks and the Metrics Monitor.

Metrics Monitor

Installed on each host in the cluster to collect system-level metrics and forward to the Metrics Collector.

Metrics Hadoop Sinks

Plugs into the various Hadoop components sinks to send Hadoop metrics to the Metrics Collector.

Architecture

Following image depicts the high level conceptual architecture of the new Ambari Metrics System.:

 

Image Removed

Key design points in the conceptual architecture:

Image Added

The Metrics Collector

...

is daemon that receives data from registered publishers

...

Additionally, the metrics collector provides ability to plugin aggregators for the collected metric data and streams data to the aggregation layer in real-time. 

...

The current system implementation does a post-write aggregation and does not require live streaming of the data to the aggregation layer.

...

Ambari Views on top of Phoenix provide ad-hoc query capability to the user along with a customized views for deployed services.

(the Monitors and Sinks). The Collector itself is build using Hadoop technologies such as HBase Phoenix and ATS. The Collector can store data on the local filesystem (referred to as "embedded mode") or use an external HDFS (referred to as "distributed mode").

Info

The entire System can be installed and managed by Ambari. The Service Definition can be found here:

https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0

Learn More

Browse the following to learn more about the Ambari Metrics REST API specification and about advanced Configuration of AMS

...

.