Page History

Problem Statement

Eagle today does not have enough dynamic capabilities to support generic metric monitoring. Generic metric monitoring needs solve the following hard problems.

• Dynamically adding metrics in existing topology. It is common that one application may emit multiple metrics and the number of metrics may change. 
• Dynamically distributing metric and policy in existing topology. With multiple metrics and multiple policies per metric, even distribution of policy compute is critical
• Dynamically joining metrics. If one policy is for multiple metrics, we should make sure those multiple metrics come into this policy

Use Case Description

From Hadoop Native Metrics Monitoring, we can observed that 

...

• Understand the metric by metadata driven (a "metricName" field in the input tuple)
• Multiplex the metrics into different stream names (This is driven by the metric name/type)

Policy Executor

...

A route table is calculated and used by Multiplexer to decide how to route the given tuple. Basically, the route table ask answers the question "Given current tuple, which stream should the multiplexer route it to."

...

So that given m1 as current tuple, it would multiplex routed to stream1 and stream2 by look at the route table. 

Metadata

The current policy definition would be extend extended to express the relationship between policy and metric. This would be used in MultiPlexer and Policy Executor to build the stream route map

...

   “Metrics”: [“m1”, “m2”] — 映射到#2的metric name 

},

{

   “policydef” : “…”

   “Metrics”: [“m1”, “m3"]

...