OBJECTIVE
An Admin of Apache Eagle should be able to get audit details of actions performed on Policies and Site/Datasource.
The Audit details should have details like User, Action(Create/Update/Delete), Timestamp, Link to the Policy/Site/Datsource etc.
PROPOSED APPROACH
Approach #1
- Request for Create/Update/Delete comes in for Policies, Site or Datasource.
- After the requested data is persisted in corresponding HBase tables, the audit information will be stored on separate audit tables (one for each of the parent tables where actual data is stored for the given service request).
- Response for the request is sent back to the user.
Approach #2
- Request for Create/Update/Delete comes in for Policies, Site or Datasource.
- After the requested data is persisted in corresponding HBase tables, the audit information will be stored on a single audit table (for any action by any service).
- Response for the request is sent back to the user.
Approach #2 tries to be more generic, as new tables definitions and implementations need not be added for auditing a new service.
PROPOSED DESIGN
Following is the initial design upon which changes will be made to accommodate the auditing feature.
For the purpose of explanation of the design, will be using Policy Definition service as an example.
#1 – Client sends a request to create a Policy to the service component with a certain payload.
http://localhost:8080/eagle-service/rest/entities?serviceName=AlertDefinitionService
#2 – After authentication and preprocessing, the call lands to the create(Entities, EntityDefinition) method in HBaseStorage.java
#3 – The required data for the operation is persisted onto HBase.
#4 – At this point, the method will build and send the response back to the client. The Auditing of the service request will be done here (implementation of one of the approaches mentioned above).
Since there are separate methods for Create, Update and Delete in HBaseStorage.java we would be able to audit what action happened for the particular request so that it can be audited.
#5 – The audit data will be persisted in a single HBase Audit table or multiple HBase Audit tables (as given in the approach)
#6 – After the audit information is persisted in HBase, the response for the service will be set back to the client.
SAMPLE TABLE DESIGN
Approach #1
// Individual audit tables for individual data tables
Service #1: Policy Definition
Data Table: alertdef
Audit Table: alertdefAudit
Audit Columns:
- encodedRowKey (encoded format of the row key as obtained from persisting the policy data)
- userID
- actionTaken (CREATE/UPDATE/DELETE)
- auditTimestamp
Service #2: Alert Data Source
Data Table: alertDataSource
Audit Table: alertDataSourceAudit
Audit Columns:
- encodedRowKey (encoded format of the row key as obtained from persisting the datasource information)
- userID
- actionTaken (CREATE/UPDATE/DELETE)
- auditTimestamp
Approach #2
// Single audit table for all data tables
Service #1: Policy Definition
Service #2: Alert Data Source
Audit Table: serviceAudit
Audit Columns:
- serviceName (to differentiate which service the audit entry belongs to)
- encodedRowKey (encoded format of the row key as obtained from persisting the datasource information)
- userID
- actionTaken (CREATE/UPDATE/DELETE)
- auditTimestamp
AUDIT RETRIEVAL APPROACH
Below are the designs for retrieving audit data for each of the proposed approaches.
APPROACH #1
As this approach suggests using multiple tables, we would need to create multiple Entity Definitions and DAO implementations for each of the audit tables created.
SAMPLE SERVICE CALL
http://localhost:8080/eagle-service/rest/list?query=AlertDefinitionServiceAudit[@encodedRowKey="ABC_DEF" AND @actionTaken="CREATE/UPDATE/DELETE"]{*}&pageSize=100
APPROACH #2
As we are going with only one table for auditing for as many numbers of data tables available, we need only one Entity Definition and DAO implementation for the single audit table created.
In the service though we would be passing only additional parameter as compared to the Approach #1 as this will be used to identify for which service the audit entries needs to be retrieved.
SAMPLE SERVICE CALL
http://localhost:8080/eagle-service/rest/list?query=AuditService[@serviceName="AlertDefinitionService" AND @encodedRowKey="ABC__DEF"]{*}&pageSize=100
http://localhost:8080/eagle-service/rest/list?query=AuditService[@serviceName="AlertDefinitionService" AND @encodedRowKey="ABC__DEF" AND @actionTaken="CREATE/UPDATE/DELETE"]{*}&]{*}&pageSize=100
Problem Statement :
As an admin of Apache Eagle I would like to know who created policies, modified policies, Deleted policies.Also who added source, site etc. This information is a must have for security related products to have an audit trail.
Solution :
To be Updated..