Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Whole topology state store. Lightweight Asynchronous Snapshots for Distributed Dataflows is valid system method to take a snapshot of stream system while not halt the stream. 
  2. Single bolt status store
    In current tech stack adoption, Siddhi already support specify persistent store to persistent the engine status. This could be done the framework level to periodically take CEP engine snapshot to hbase store. Bolt status store including CEP engine and outside-engine status(e.g. policy acceptance related information)


Recovery

For the stored state

  1. Whole topology state would be used to restore when a topology restarted.
  2. Single bolt status restore:
    When a bolt is started, it would try to load from the snapshot store where the snapshot match the current bolt's policy acceptance. 

 

Historical data

This related to the how analytic runtime handle the long time window data. A supportive analytic runtime should be able to embrace the pre-load of historical data since the framework should be able to call the runtime to load/store snapshots at correct time.

 

This state management could be complete topic its won. It's now covered at Policy State Management.

 

Persistence

As a general monitoring tool, eagle not mean to store even point of metric into storage, user have to define a time window to reduce the gratuity of the metrics. This is user written CEP-QL above.

...