Problem

If user wants to use Eagle for general metric/log-based monitoring like hadoop jmx monitoring, user has to setup-up development environment, rewrite code though we provided high-level fluent DSL and re-build, package and submit the topology manually, which is not good for user experience nor re-use eagle's existing capability for more monitoring cases, the better way is to allow user to dynamically define monitoring topology in very light-weight way, so we propose a dynamical monitoring framework.

Features 

The dynamical Dynamical Monitoring Framework is consist of four parts:  

• Pipeline Builder: Dynamical   Declarative Pipeline Modeling, DSL, Parser and Compiler
• Pipeline Runner:  Provide tool/CLI to read pipeline definition, parse, compile and submit pipeline
• Pipeline Scheduler: Automatically pipeline scheduler
• Pipeline Designer: Graphic-baed Pipeline designer

...

•  Compile DSL Configure to Pipeline model

...

•  Fix configuration conflict

Components

Dynamical Pipeline Builder

Pipeline ::= config + schema + dataflow

...

Finally compile to follow DAG in runtime.

Dynamical Pipeline Runner

•  Pipeline Runner CLI

Dynamical Pipeline Scheduler

Dynamical Pipeline Designer

Requirements

• User could define the monitoring DAG pipeline in UI by DSL
• User could define the monitoring DAG pipeline in UI with UI actions 

Use Case

Hadoop JMX Metric Monitoring

...