Versions Compared

Old Version 4

changes.mady.by.user puyulu

Saved on

compared with

New Version Current

changes.mady.by.user puyulu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Application "TOPOLOGY HEALTH CHECK" provide embedded collector script to ingest hadoop/hbase jmx metric as eagle stream and provide ability to define alert policy and detect anomaly in real-time from metricaims to monior those servies with a master-slave structured topology and provide metrics at host level.

 

TypeTOPOLOGY_HEALTH_CHECK
Version0.5.0-version
DescriptionCollect MR,HBASE,HDFS node status and cluster ratio
StreamsTOPOLOGY_HEALTH_CHECK_STREAM
Configuration
  • Topology Health Check Topic (default: topology_health_check)
  • Kafka Broker List (default: sandobox.hortonworks.com:6667)

...

  1. Make sure already setup a site (here use a demo site named "sandbox")
  2. Install "Hadoop JMX MonitorTopology Health Check" app in eagle server Image Removed
    Image Added
  3. Configure Application settings Image Removed
    Image Added
  4. Ensure a kafka topic named topology_health_check (In current guide, it should be topology_health_check)Setup metric collector for monitored Hadoop/HBase using hadoop_jmx_collector and modify the configurationCollector scripts: https://github.com/apache/incubator-eagle/tree/master/eagle-external/hadoop_jmx_collector

  5. Rename config-sample.json to config.json: https://github.com/apache/incubator-eagle/blob/master/eagle-external/hadoop_jmx_collector/config-sample.json

    Code Block
    languagejs
    titleconfig.json
    collapsetrue
    {
	env: {
		site: "sandbox",
		name_node: {
			hosts: [
				"sandbox.hortonworks.com"
			],
			port: 50070,
			https: false
		},
		resource_manager: {
			hosts: [
				"sandbox.hortonworks.com"
			],
			port: 50030,
			https: false
		}
	},
	inputs: [{
		component: "namenode",
		host: "server.eagle.apache.org",
		port: "50070",
		https: false,
		kafka_topic: "nn_jmx_metric_sandbox"
	}, {
		component: "resourcemanager",
		host: "server.eagle.apache.org",
		port: "8088",
		https: false,
		kafka_topic: "rm_jmx_metric_sandbox"
	}, {
		component: "datanode",
		host: "server.eagle.apache.org",
		port: "50075",
		https: false,
		kafka_topic: "dn_jmx_metric_sandbox"
	}],
	filter: {
		monitoring.group.selected: [
			"hadoop",
			"java.lang"
		]
	},
	output: {
		kafka: {
			brokerList: [
				"localhost:9092"
			]
		}
	}
}
  6. Click "Install" button then you will see the TOPOLOGY_HEALTH_CHECK_STREAM_{SITE_ID} in Streams Image Removed

Usage

...

  1. Image Added

Define Health Check Alert Policy

 

  1. Go to "Define Policy"
  2. Select TOPOLOGY_HEALTH_CHECK related streams

  3. Define SQL-Like policy, for example 

    Code Block
    languagesql
    from TOPOLOGY_HEALTH_CHECK_STREAM_SANDBOX[status=='dead'] select * insert into topology_health_check_stream_out;

     Image Added

 

   Image Removed