THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Introduction
Application "HADOOP_JMX_METRIC_MONITORTOPOLOGY HEALTH CHECK" provide embedded collector script to ingest hadoop/hbase jmx metric as eagle stream and provide ability to define alert policy and detect anomaly in real-time from metric.
Type | HADOOPTOPOLOGY_JMXHEALTH_METRIC_MONITORCHECK |
---|---|
Version | 0.5.0-version |
Description | Collect JMX Metric and monitor in real-timeMR,HBASE,HDFS node status and cluster ratio |
Streams | HADOOPTOPOLOGY_JMXHEALTH_METRICCHECK_STREAM |
Configuration |
|
Setup & Installation
- Make sure already setup a site (here use a demo site named "sandbox")
- Install "Hadoop JMX Monitor" app in eagle server
- Configure Application settings
- Ensure a kafka topic named
hadooptopology_jmx_metric_{SITE_ID} health_check (In current guide, it should be hadoop
topology_
jmxhealth_
metric_sandboxcheck
) - Setup metric collector for monitored Hadoop/HBase using hadoop_jmx_collector and modify the configuration
- Collector scripts: https://github.com/apache/incubator-eagle/tree/master/eagle-external/hadoop_jmx_collector
Rename config-sample.json to config.json: https://github.com/apache/incubator-eagle/blob/master/eagle-external/hadoop_jmx_collector/config-sample.json
Code Block language js title config.json collapse true { env: { site: "sandbox", name_node: { hosts: [ "sandbox.hortonworks.com" ], port: 50070, https: false }, resource_manager: { hosts: [ "sandbox.hortonworks.com" ], port: 50030, https: false } }, inputs: [{ component: "namenode", host: "server.eagle.apache.org", port: "50070", https: false, kafka_topic: "nn_jmx_metric_sandbox" }, { component: "resourcemanager", host: "server.eagle.apache.org", port: "8088", https: false, kafka_topic: "rm_jmx_metric_sandbox" }, { component: "datanode", host: "server.eagle.apache.org", port: "50075", https: false, kafka_topic: "dn_jmx_metric_sandbox" }], filter: { monitoring.group.selected: [ "hadoop", "java.lang" ] }, output: { kafka: { brokerList: [ "localhost:9092" ] } } }
- Click "Install" button then you will see the
HADOOPTOPOLOGY_JMXHEALTH_METRICCHECK_STREAM_{SITE_ID}
in Streams
Usage
...
Define Health Check Alert Policy
- Go to "Define Policy"
- Select HADOOPTOPOLOGY_JMXHEALTH_METRIC_MONITOR CHECK related streams
Define SQL-Like policy, for example
Code Block language sql from HADOOPTOPOLOGY_JMXHEALTH_METRICCHECK_STREAM_SANDBOX[metric=="cpu.usage" and value > 0.9] select site,host,component,value status=='dead'] select * insert into HADOOPtopology_CPUhealth_USAGEcheck_GTstream_90_ALERTout;