THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Problem Statement
Eagle Topology failed to Start if it finds invalid Policy.
Proposed Design
Identifying an invalid query (using SiddhiCompiler)
AlertDefinitionAPIEntity class is used to define policy.
From the request payload, the policyDef tag contains the values of the type of the policy and the actual policy (given in bold below)
[{
“tags”: {
“site”: “sandbox”, “dataSource”: “hdfsAuditLog”, “policyId”: “testPolicy”, “alertExecutorId”: “hdfsAuditLogAlertExecutor”, “policyType”: “siddhiCEPEngine”
},
“desc”: “test alert policy”,
"policyDef": "{"type":"siddhiCEPEngine","expression":"from hdfsAuditLogEventStream[src =='/tmp/private'] select * insert into outputStream;"}",
“notificationDef”: “[{
“sender”:”noreply-eagle@company.com”, “recipients”:”user@company.com”, “subject”:”test alert policy”, “flavor”:”email”, “id”:”email_1”
}]”,
“enabled”: true
}]
The process of finding out whether a policy expression is valid/invalid can be achieved by using SiddhiCompiler.parseQuery().
The method will check if the given input expression is valid wrt expected syntax. If the policy is found to be invalid, it will throw SiddhiParserException with mentioning the error in the query. Based on this method output, we can define markdown status and markdown reason for the query corresponding to its policy.
Updating Markdown details for the Policy in HBase
Approach #1 (will not be used as this makes Siddhi a dependency in the HBase data persistence flow)
From the HBaseStorage class, whenever a request comes in to persisted Policy details (request under the service name AlertDefinitionService), we could put a check on the query to see whether it is valid syntax WRT to Siddhi. This can be achieved using SiddhiCompiler.parseQuery(String query). Based on the output, we can add another field, say queryCompiled and yet another field for error message if the query is not compiled.
Based on the above result, we can modify the inbound entities by adding markdown and markdownReason column values before persisting them.
Approach #2
Considering the policies are already persisted to HBase at this point before starting a topology.
When the topology starts, the AlertExecutor.init() will be called to populate the map of PolicyEvaluators. The map is updated regularly at one of these methods - init(), onPolicyCreated(), onPolicyChanged() and onPolicyDeleted() . At this point before making any change to the map, we could check for the validity of the policy and then update it accordingly in the HBase tabled against the policyID. Also based on the output of the validation process, we can add/update/remove AlertExecutors from the map which will be used later in the flow to validate the incoming data event against the required policies.