...

JIRA : SQOOP-1834 and its sub tickets, SQOOP-2048 and its sub tickets

Problem

Sqoop 2 needs a pluggable role based access controller (RBAC), which is responsible for the authorization to Sqoop 2 resources, such as server, connector, link, job, submission etc.

Basic Idea

• The access controller is pluggable

...

Resource, actions and rules

Server has three children: Connector, Link, Job.

• It is a hierarchy mode. If a user has the privilege of {server, all}, then he/she has all privileges of {connector, all}, {link, all} and {job, all}.
• If a user has the privilege of {job, all}, then he/she has both privileges of {job, read} and {job, write}.
• If a user want to create a link, then he/she need to have the privilege of {server, create}

Authorization framework

...

#org.apache.sqoop.authorization.handler=org.apache.sqoop.security.DefaultAuthorizationHandler
#org.apache.sqoop.authorization.controller=org.apache.sqoop.security.DefaultAccessController
#org.apache.sqoop.authorization.validator=org.apache.sqoop.security.DefaultAuthorizationValidator

Image RemovedImage Added

• Four metadata classes.
  • Role
  • principal
    • This class defines user or group.
    • Type: user, group, role.
    • principal could be granted a role. i.e. if we want to grant a admin role to user hadoop, then grantRole (principal (name=hadoop, type=user), role (name=admin)).
  • Resource
    • This class defines four resources in Sqoop 2.
    • Type: server, connector, link, job, submission.
  • Privilege
    
    • Action: createall, view, update, delete, use, enable, disableread, write.
    • with_grant_option: boolean, defines whether the role could grant this privilege to other role.

...

Override
public void createLinkPrivilige() throws SqoopAccessControlException {
    List<Privilege> privileges;
    privileges.add(new Privilege(new Resource("Link", "1"), "Create", null));
    privileges.add(new Privilege(new Resource("Connector", "1"), "UseRead", null));
    AuthorizationManager.getAuthenticationHandler.checkPrivileges(privileges);
}

...

• The grant/revoke privilege should be run in command line in Sqoop client
• The commands are showed below

Create/Drop Role

show role
show role -CREATE ROLE role_name admin
show

DROP ROLE role -principal_name sqoop -principal_type user
show role -privilege_name my_privilege
add role –role_name admin
update role -old_role_name admin -new_role_name developer
remove role -role_name admin
 
show principal
show principal -principal_name sqoop
show principal -principal_type user
show principal -principal_name sqoop -principal_type user
show principal -role_name admin
add principal –principal_name sqoop -principal_type user
update principal -old_principal_name sqoop -old_principal_type user -new_principal_name hadoop -new_principal_type group
remove principal -principal_name sqoop
remove principal -principal_type user
remove principal -principal_name sqoop -principal_type user


grant role –role_name admin -principal_name sqoop -principal_type user
revoke role –role_name admin -principal_name sqoop -principal_type user
 
show resource
show resource -resource_type link
show resource -resource_name 1 -resource_type link
add resource -resource_name 1 -resource_type link
update resource -old_resource_name 1 -old_resource_type link -new_resource_name 2 -new_resource_type link
remove resource -resource_type link
remove resource -resource_name 1 -resource_type link
 
show privilege
show privilege -privilege_name my_privilege
show privilege -resource_name 1 -resource_type link
show privilege -role_name admin
add privilege -resource_name 1 -resource_type link
update privilege -old_resource_name 1 -old_resource_type link -old_action create -old_with_grant_option true -new_resource_name 1 -new_resource_type link -new_action create -new_with_grant_option true
remove privilege -privilege_name my_privilege
remove privilege -resource_name 1 -resource_type link
remove privilege -resource_name 1 -resource_type link -action create -with_grant_option true
 
grant privilege –role_name admin -privilege_name my_privilege
revoke privilege –role_name admin -privilege_name my_privilege
 
SHOW ROLE

Grant/Revoke Roles

GRANT ROLE role_name [, role_name] ... TO principal_specification [, principal_specification] ...

REVOKE ROLE role_name [, role_name] ... FROM principal_specification [, principal_specification] ...
principal_specification:
    USER user_name | GROUP group_name | ROLE role_name

Viewing Granted Roles

SHOW ROLE GRANT principal_specification

SHOW PRINCIPAL ON ROLE role_name
 
principal_specification:
    USER user_name | GROUP group_name | ROLE role_name

Grant/Revoke Privileges

GRANT privilege_action_type [, privilege_action_type] ... ON resource [, resource] ... TO principal_specification [, principal_specification] ... [WITH GRANT OPTION]

REVOKE [GRANT OPTION FOR] privilege_action_type [, privilege_action_type] ... ON resource [, resource] ... FROM principal_specification [, principal_specification] ...

REVOKE ALL PRIVILEGES FROM principal_specification [, principal_specification] ...

privilege_action_type:
    ALL | CREATE | READ | WRITE
 
resource:
    SERVER server_name | CONNECTOR connector_name | LINK link_name | JOB job_name
 
principal_specification:
    USER user_name | GROUP group_name | ROLE role_name

Viewing Granted Privileges

SHOW GRANT principal_specification [ON resource]
 
principal_specification:
    USER user_name | GROUP group_name | ROLE role_name
 
resource:
    SERVER server_name | CONNECTOR connector_name | LINK link_name | JOB job_name

• Restful call API is handled by org.apache.sqoop.handler.AuthorizationEngine.java in sqoop-server
  • GET POST /v1authorization/role/{role_name}
    • Return details about one particular role with role_name
    • Return all of them if role_name is null
  • GET /v1/role/principal/name/{name}/type/{type}
    
    • Return all roles the particular principal has with principal (name, type)
  • GET /v1/role/privilege/{privilege_name}
    
    • Return all roles with the particular privilege (privilege_name)
  • PUT /v1/role
    • Create new role with role_name. Put data of JsonObject role (role_name)
    • Update existing role with old_role_name and new_role_name. Put data of JsonObject old_role (role_name) and new_role (role_name)
  DELETE /v1/role/{role_name}
  
  • GET /v1/principal/name/{name}/type/{type}
    • Return details about one particular principal with name and type
    • Return all of them if name is null or type is null or both are null
  • GET /v1/principal/role/{role_name}
    • Return all principals with the particular role (role_name)
  • PUT /v1/principal
    • Create new principal with name and type. Put data of JsonObject principal (name, type)
    • Update existing principal with old_name, old_type, new_name, new_type. Put data of JsonObject old_principal (name, type) and new_principal (name, type)
  • DELETE /v1/principal/name/{name}/type/{type}
    
    • Delete all of them if name is null or type is null
  • roles/create
    
    • Create new role with {name}
  • DELETE /authorization/role/{role-name}
    
    
  • GET /authorization/roles
    • Show all roles
  • GET /authorization/principals?role_name={name}
    
    • Show all principals in role with {name}
  • GET /authorization/roles?principal_type={type}&principal_name={name}
    
    • Show all roles in principal with {name, type}
  • PUT /authorization/roles/grantPUT /v1/grant_role
    • Grant a role to a user/group/role
    • PUT data of JsonObject role(role_name) and principal (name, type)
  • PUT /authorization/v1roles/revoke_role
    • Revoke a role to a user/group/role
    • PUT data of JsonObject role(role_name) and principal (name, type)
    GET
  • PUT /v1authorization/resource/name/{name}/type/{type}
    • Return details about one particular resource with name and type
    • Return all of them if name is null or type is null or both are null
    privileges/grant
    • Grant a privilege to a principal
    • PUT data of JsonObject principal
    PUT /v1/resource
    
    • Create new resource with name and type. Put data of JsonObject resource (name, type)
    • Update existing resource with old_name, old_type, new_name, new_type. Put data of JsonObject old_resource (name, type) and new_resource (name, type)
  • DELETE /v1/resource/name/{name}/type/{type}
    
    • Delete all of them if name is null or type is null
  • GET /v1/privilege/{privilege_name}
    • Return details about one particular privilege with privilege_name
    • Return all of them if name is null or type is null or both are null
  • GET /v1/privilege/resource/name/{name}/type/{type}
    • Return all privilege with the particular principal (name, type)
  • GET /v1/privilege/role/{role_name}
    • Return all privilege with the particular role (role_name)
  • PUT /v1/privilege
    • Create new privilege with resource, action and with_grant_option. Put data of JsonObject resource (name, type), action and with_grant_option
    • Update existing privilege with old_resource, old_action, old_with_grant_option, new_resource, new_action and new_with_grant_option. Put data of JsonObject old_privilege (resource (name, type), action, with_grant_option) and new_privilege (resource (name, type), action, with_grant_option)
  • DELETE /v1/privilege/{privilege_name}
  • DELETE /v1/resource/name/{name}/type/{type}
  DELETE /v1/resource/name/{name}/type/{type}/action/{action}/with_grant_option/{with_grant_option}
  
  • PUT /v1/grant_privilege
    • Grant a privilege to a role
    • PUT data of JsonObject role(role_name) and privilege (name)
  • PUT /v1/revoke_privilege
    • Revoke a privilege to a role
    • PUT data of JsonObject role(role_name) and privilege (name)
  • •  and privilege (resource-name, resource-type, action, with-grant-option)
  • PUT /authorization/privileges/revoke
    • Revoke a privilege to a principal
    • PUT data of JsonObject principal(name, type) and privilege (resource-name, resource-type, action, with-grant-option)
    • If privilege is null, then revoke all privileges for principal(name, type)
  • GET /authorization/privileges?principal_type={type}&principal_name={name}&resource_type={type}&resource_name={name}
    • Show all privileges in principal with {name, type} and resource with {resource-name, resource-type}
    • If resource is null, then show all privileges in principal with {name, type}

Sentry implementation

• Sentry could be used as an alternative access controller
• Config in sqoop.properties

...

• Role table
  • Id
  • Name
  • Comment
    • Role name could be admin, developer, user, etc.
• Role_User_Group table
  • Id
  • Role_id
  • User_name
  • Group_name
  • Comment
    • The information of user and group comes from Linux or LDAP etc.
    • Only one of user name and group name is set. If user name is set and leave group name empty, it means that this user has this rule. If group name is set and leave user name empty, it means that all users in this group has this rule.
    • One user/group could have one or multiple roles.
• Privilege table
  • Id
  • Role_id
  • Resource_id
  • Resource_type
  • Action_type
  • Comment
    • Resource type could be the existing resource table, such as connector, link, job, submission, etc.
    • Resource type could be added in the future, say config etc.
    • If resource_id is 0, it means all resource of this type, ie. resource_id=0 and resource_type=link means all links.
    • Use resource id and resource type to identify the resource, ie. resource_id=1 and resource_type=link means the resource of “select * from link where id =1”.
    • Action type could be read, create, update, delete, use etc.
• Accordingly, MRole, MRoleUserGroup and MPrivilege classes are added into package org.apache.sqoop.model.

...