Page History

...

1. Design tables: metricSchemaDef, metricGroupDef, policyDef
2. Storm topology: one spout + N correlation bolts
3. Spout
   1. reference KafkaSpout, but may need some fundamental changes if we want support multiple topic
   2. maintain a route table which maps metricGroup to correlation bolt ID. This table should be identical across all spout instances
   3. read metricGroupDef and metricSchemaDef in background and spawn new thread to read new metric
4. Correlation bolt
   1. read policyDef

Engineering Design

Implementations:

1. Example code is under https://github.com/yonzhang/incubator-eagle/tree/hackillinois
   1. eagle-examples/eagle-correlation-engine   topology example to wrap multiple KafkaSpout into one spout
   2. eagle-examples/eagle-correlation-service   service example to provide API to read/write metadata, for example metric, metric group etc.