Page History

...

Therefore, we want to support spark-sql syntax directly, as one of griffin DSL.

Griffin DSL of

...

version 0.2.0

In griffin job configure file, the "evaluateRule" field will be like this to support griffin DSL:

...

In the backend, griffin will execute the sql statements, and persist the metric and record if configured.

Griffin DSL of version 0.2.1

In dq config file of griffin in new version 0.2.1, there're some modification in the "evaluateRule" field, wrapping "metric" and "record" filed inside "out" array:

"evaluateRule": {
  "rules": [
    {
      "dsl.type": "griffin-dsl",
      "dq.type": "accuracy",
      "name": "accu",
      "rule": "source.name = target.name AND source.age = target.age",
      "details": {
        "source": "source",
        "target": "target"
      },
      "out": [
        {
          "type": "metric",
          "name": "accu"
        },
        {
          "type": "record",
          "name": "missRecords"
        }
      ]
    }
  ]
}

Griffin Measure Process Design

...