Introduction

Application "TOPOLOGY HEALTH CHECK" provide embedded collector script to ingest hadoop/hbase jmx metric as eagle stream and provide ability to define alert policy and detect anomaly in real-time from metric.

Setup & Installation

1. Make sure already setup a site (here use a demo site named "sandbox")
2. Install "Topology Health Check" app in eagle server
3. Configure Application settings
4. Ensure a kafka topic named topology_health_check (In current guide, it should be topology_health_check)
5. Setup metric collector for monitored Hadoop/HBase using hadoop_jmx_collector and modify the configuration
   1. Collector scripts: https://github.com/apache/incubator-eagle/tree/master/eagle-external/hadoop_jmx_collector

   2. Rename config-sample.json to config.json: https://github.com/apache/incubator-eagle/blob/master/eagle-external/hadoop_jmx_collector/config-sample.json

      config.json Expand source

      {
      	env: {
      		site: "sandbox",
      		name_node: {
      			hosts: [
      				"sandbox.hortonworks.com"
      			],
      			port: 50070,
      			https: false
      		},
      		resource_manager: {
      			hosts: [
      				"sandbox.hortonworks.com"
      			],
      			port: 50030,
      			https: false
      		}
      	},
      	inputs: [{
      		component: "namenode",
      		host: "server.eagle.apache.org",
      		port: "50070",
      		https: false,
      		kafka_topic: "nn_jmx_metric_sandbox"
      	}, {
      		component: "resourcemanager",
      		host: "server.eagle.apache.org",
      		port: "8088",
      		https: false,
      		kafka_topic: "rm_jmx_metric_sandbox"
      	}, {
      		component: "datanode",
      		host: "server.eagle.apache.org",
      		port: "50075",
      		https: false,
      		kafka_topic: "dn_jmx_metric_sandbox"
      	}],
      	filter: {
      		monitoring.group.selected: [
      			"hadoop",
      			"java.lang"
      		]
      	},
      	output: {
      		kafka: {
      			brokerList: [
      				"localhost:9092"
      			]
      		}
      	}
      }

6. Click "Install" button then you will see the TOPOLOGY_HEALTH_CHECK_STREAM_{SITE_ID} in Streams

Usage

Define Health Check Alert Policy

1. Go to "Define Policy"
2. Select TOPOLOGY_HEALTH_CHECK related streams

3. Define SQL-Like policy, for example 

   from TOPOLOGY_HEALTH_CHECK_STREAM_SANDBOX[status=='dead'] select * insert into topology_health_check_stream_out;