THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Old Griffin architecture problems
What's the painpoints pain points we are facing in the current edition of Griffin from the architectural perspective?
- To have an abstraction layer of query engine
The current spark implementation limited Griffin's adoption, since a company could use other query engine in company level. It's too heavy to set up a spark environment to run Griffin. - To support a common data quality workflow: measure-monitorevaluate-alert
In most of real scenarios, measuring is not the goal, but measure-monitorevaluate-alert workflow.
user should be able to define an use case, including:
1. to measure a dq data quality metrics
2. to evaluate the dq data quality trigger
3. to define the alerting action
4. to integrate the 3-steps into a single job/UoW (by scheduler)
...