...
JIRA : SQOOP-1834 and its sub tickets, SQOOP-2048 and its sub tickets
Problem
Sqoop 2 needs a pluggable role based access controller (RBAC), which is responsible for the authorization to Sqoop 2 resources, such as server, connector, link, job, submission etc.
Basic Idea
- The access controller is pluggable
...
Resource, actions and rules
Server has three children: Connector, Link, Job.
- It is a hierarchy mode. If a user has the privilege of {server, all}, then he/she has all privileges of {connector, all}, {link, all} and {job, all}.
- If a user has the privilege of {job, all}, then he/she has both privileges of {job, read} and {job, write}.
- If a user want to create a link, then he/she need to have the privilege of {server, create}
Resource | Global Namespace | |
---|---|---|
Server |
| |
Connector |
| |
Link |
| |
Job |
| |
Resource | Global Namespace | Instance |
Connector |
|
|
Link |
|
|
Job |
|
|
Submission |
|
|
Action | Privilege needed | ||
---|---|---|---|
show connector |
| ||
show link |
| ||
create link |
| ||
update link |
| ||
delete link |
| ||
enable link |
| ||
disable link |
| ||
show job |
| ||
create job |
| ||
update job |
| ||
delete job |
| ||
enable job |
| ||
disable job |
| ||
show submission |
| ||
create submission (start job) |
| ||
update submission (stop job) |
| ||
| |||
start job |
| ||
stop job |
| ||
show submission |
| delete submission |
|
Authorization framework
...
Code Block |
---|
#org.apache.sqoop.authorization.handler=org.apache.sqoop.security.DefaultAuthorizationHandler #org.apache.sqoop.authorization.controller=org.apache.sqoop.security.DefaultAccessController #org.apache.sqoop.authorization.validator=org.apache.sqoop.security.DefaultAuthorizationValidator |
- Three Four metadata classes.
- Role
- principal
- This class defines user or group.
- Type: user, group, role.
- principal could be granted a role. i.e. if we want to grant a admin role to user hadoop, then grantRole (principal (name=hadoop, type=user), role (name=admin)).
- Resource
- This class defines four resources in Sqoop 2.
- Type: server, connector, link, job, submission.
- Privilege
- Action: createall, view, update, delete, use, enable, disableread, write.
- with_grant_option: boolean, defines whether the role could grant this privilege to other role.
- Five classes will be added into Sqoop-core as org.apache.sqoop.security package.
- AuthorizationManager
- Similar with other Sqoop Manager, ie. ConnectorManager, RepositoryManager, etc., the AuthorizationManager handles two singleton instances, AuthorizationManager and AuthorizationHandler.
- The initialize function is run when starting the Sqoop server
- The initialize function will initial AuthorizationHandler, according to the handler name (DefaultAuthorizationhandler or SentryAuthorizationHandler) from configuration file (sqoop.properties).
- AuthorizationHandlerFactory
- It is a factory design mode.
- It is to use ClassUtils.loadClass to refact the real AuthorizationHandler in getAuthorizationHandler function.
- AuthorizationHandler
- It is an abstract class.
- There is a default implementation (DefaultAuthorizationHandler) in Sqoop-security component.
- It handles two singleton instances, AccessController and AuthorizationValidator.
- All function will be delegated to these two instances to handle. AccessController to handle grantRole, revokeRole, grantPrivilege and revokePrivilege. AuthorizationValidator to handle checkPrivilege.
- AccessController
- It is an abstract class.
- There is a default implementation (DefaultAccessController) in Sqoop-security component.
- This class is responsible to manage roles, privileges.
- AuthorzationValidator
- It is an abstract class.
- There is a default implementation (DefaultAuthorizationValidator) in Sqoop-security component.
- This class is responsible to check privileges.
- AuthorizationManager
- Three classes will be added into Sqoop-security as org.apache.sqoop.security package.
- DefaultAuthorizationHandler
- This class extends abstract AuthorizationHandler.
- It handles two singleton instances, DefaultAccessController and DefaultAuthorizationValidator.
- DefaultAccessController
- This class extends abstract AccessController.
- Default AuthorzationValidator
- This class extends abstract AuthorizationValidator.
- As default/simple implementation, it always returns true and will not check the privilege actually.
- DefaultAuthorizationHandler
- All functions in RequestHandler, which handles all requests, ie. create link, will be added privilege validation check.
...
Code Block |
---|
Override public void createLinkPrivilige() throws SqoopAccessControlException { List<Privilege> privileges; privileges.add(new Privilege(new Resource("Link", "1"), "Create", null)); privileges.add(new Privilege(new Resource("Connector", "1"), "UseRead", null)); AuthorizationManager.getAuthenticationHandler.checkPrivileges(privileges); } |
...
- The grant/revoke privilege should be run in command line in Sqoop client
- The commands are showed below
Create/Drop Role
Code Block |
---|
showCREATE ROLE role show role -pid 1 update role -rid 1 -name admin remove role -rid 1 grant role –name user add role –id 1 –name user remove role –id 1 show role_user_group grant principal –role_id 1 –user_name sqoop grant principal –role_id 1 –group_name sqoop revoke principal –role_id 1 –user_name sqoop revoke principal –role_id 1 –group_name sqoop show privilege grant privilege –resource_type link –resource_id 1 –role_id 1 –action_type read -with-grant-option yes revoke privilege –resource_type link –resource_id 1 –role_id 1 –action_type read_name DROP ROLE role_name SHOW ROLE |
Grant/Revoke Roles
Code Block |
---|
GRANT ROLE role_name [, role_name] ... TO principal_specification [, principal_specification] ...
REVOKE ROLE role_name [, role_name] ... FROM principal_specification [, principal_specification] ...
principal_specification:
USER user_name | GROUP group_name | ROLE role_name |
Viewing Granted Roles
Code Block |
---|
SHOW ROLE GRANT principal_specification
SHOW PRINCIPAL ON ROLE role_name
principal_specification:
USER user_name | GROUP group_name | ROLE role_name |
Grant/Revoke Privileges
Code Block |
---|
GRANT privilege_action_type [, privilege_action_type] ... ON resource [, resource] ... TO principal_specification [, principal_specification] ... [WITH GRANT OPTION]
REVOKE [GRANT OPTION FOR] privilege_action_type [, privilege_action_type] ... ON resource [, resource] ... FROM principal_specification [, principal_specification] ...
REVOKE ALL PRIVILEGES FROM principal_specification [, principal_specification] ...
privilege_action_type:
ALL | CREATE | READ | WRITE
resource:
SERVER server_name | CONNECTOR connector_name | LINK link_name | JOB job_name
principal_specification:
USER user_name | GROUP group_name | ROLE role_name |
Viewing Granted Privileges
Code Block |
---|
SHOW GRANT principal_specification [ON resource]
principal_specification:
USER user_name | GROUP group_name | ROLE role_name
resource:
SERVER server_name | CONNECTOR connector_name | LINK link_name | JOB job_name |
- Restful call API is handled by org.apache.sqoop.handler.AuthorizationRequestHandlerAuthorizationEngine.java in sqoop-server
- GET POST /v1/role/{rid}
- GET /v1/role/principal/type/{type}/name/{name}
- Return details about one particular role with id:rid
- Return all roles the particular principal has with principal (type, name)
- Return all of them if rid equals to "all"
- PUT /v1/role DELETE /v1/role/{rid}
- GET /v1/principal/{pid}
- GET /v1/principal/role/{rid}
- Return details about one particular principal with id:pid
- Return all principals the particular role has with role (rid)
- Return all of them if pid equals to "all"
- GET /v1/role/{rid}
- authorization/roles/create
- Create new role with {name}
- DELETE /authorization/role/{role-name}
- GET /authorization/roles
- Show all roles
- GET /authorization/principals?role_name={name}
- Show all principals in role with {name}
- GET /authorization/roles?principal_type={type}&principal_name={name}
- Show all roles in principal with {name, type}
- PUT /authorization/roles/grantGET /v1/role/principal/type/{type}/name/{name}
- Return details about one particular role with id:rid
- Return all roles the particular principal has with principal (type, name) Return all of them if rid equals to "all"
- Grant a role to a user/group/role
- PUT data of JsonObject role(idname) list and and principal (name, type, name) list
PUT /v1/grant_role - PUT /v1authorization/roles/revoke_role
- Revoke /Remove a existing role granta role to a user/group/role
- PUT data of JsonObject role(idname) list and principal (name, type, name) list
- PUT /v1authorization/resourceprivileges/{rid}
- Return details about one particular role with id:rid
- Return all of them if rid equals to "all"
- PUT /v1/resource DELETE /v1/resource/{rid}
- GET /v1/privilege/{pid}
- Return details about one particular privilege with id:pid
- Return all of them if pid equals to "all"
- PUT /v1/privilege DELETE /v1/privilege/{pid}
- PUT /v1/grant_privilege
- Grant a privilege to a role
- PUT data of JsonObject role(id) list and privilege (resource, action, with_grant_option) list
- grant
- Grant a privilege to a principal
- PUT data of JsonObject principal(name, type) and privilege (resource-name, resource-type, action, with-grant-option)
- PUT /authorization/privileges/revoke
- Revoke a privilege to a principal
- PUT data of JsonObject principal(name, type) and privilege (resource-name, resource-type, action, with-grant-option)
- If privilege is null, then revoke all privileges for principal(name, type)
- GET /authorization/privileges?principal_type={type}&principal_name={name}&resource_type={type}&resource_name={name}
- Show all privileges in principal with {name, type} and resource with {resource-name, resource-type}
- If resource is null, then show all privileges in principal with {name, type}
- Revoke/Remove a existing privilege grant
- PUT data of JsonObject role(id) list and privilege (resource, action, with_grant_option) list
Sentry implementation
- Sentry could be used as an alternative access controller
- Config in sqoop.properties
...
- Use Sentry to check access privilege
- Set access privilege using hue (optional)
Database design
- Role table
- Id
- Name
- Comment
- Role name could be admin, developer, user, etc.
- Role_User_Group table
- Id
- Role_id
- User_name
- Group_name
- Comment
- The information of user and group comes from Linux or LDAP etc.
- Only one of user name and group name is set. If user name is set and leave group name empty, it means that this user has this rule. If group name is set and leave user name empty, it means that all users in this group has this rule.
- One user/group could have one or multiple roles.
- Privilege table
- Id
- Role_id
- Resource_id
- Resource_type
- Action_type
- Comment
- Resource type could be the existing resource table, such as connector, link, job, submission, etc.
- Resource type could be added in the future, say config etc.
- If resource_id is 0, it means all resource of this type, ie. resource_id=0 and resource_type=link means all links.
- Use resource id and resource type to identify the resource, ie. resource_id=1 and resource_type=link means the resource of “select * from link where id =1”.
- Action type could be read, create, update, delete, use etc.
- Accordingly, MRole, MRoleUserGroup and MPrivilege classes are added into package org.apache.sqoop.model.
...