THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
search
attachments
weblink
advanced
Overview
Content Tools
Apps
livy integration
Griffin uses livy as job-submitter of spark, all jobs will be submitted via livy client.
In code base, we wrap livy client into LivyTaskSubmitHelper.java .
SparkSubmitJob.java leverages to quartz scheduler and call livy client to submit jobs
DSL parser
Griffin gives a new sql-like language to define data quality computation, there is an implementation in measure project.
GriffinDslParser.scala is responsible for parsing rules in data quality.
Sink
Data sink is a term used to describe a computer or any other medium capable of receiving data.
Sink.scala is an abstract data sink type in griffin which deals with all data processed.