THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- To have an abstraction layer of query engine
The current spark implementation limited Griffin's adoption, since a company could use other query engine in company level. It's too heavy to set up a spark environment to run Griffin. - To support a common data quality workflow: measure-monitor-alert
In most of real scenarios, measuring is not the goal, but measure-monitor-alert workflow.
user should be able to define an use case, including:
1. to measure a dq metrics
2. to evaluate the monitoring dq trigger
3. to define the alerting action
4. to integrate the 3-steps into a single job/UoW (by scheduler)
...