Hive Authorization

Introduction

Hive provides multiple ways to authorize users:

• Client authorization (default Hive authorization)  This is the first and default authorization implementation. It can be enabled in hive command line or hive server2. However, when authorization is enabled in hive client, it can easily be disabled by the user. There is also the problem that authorization model used by this mode is incomplete, and as a result has some serious gaps in security. For example, there is no restriction on which user can run a grant statement.
• Metastore server security (storage-based authorization) HDFS permissions provides access control for the data. Storage based authorization in hive metastore provides access control for the hive metadata. This authorization mode provides secure authorization for hive commandline, hcatalog users such as pig and mapreduce, and hive server2 users (when configured to run queries as the end user). However, the access control is at the database/table/partition level. It does not provide row or column level access control.
• SQL standards based authorization (introduced in Hive 0.13.0) Provides access control to users who run query via HiveServer2. It provides fine grained access control at row and column level through the use of views. It is not supported in hive commandline. Use of storage based authorization is reccomended in conjunction with this mode, to provide authorization for hive commandline or hcatalog (pig/mapreduce) use cases.

Hive Client Authorization

This section describes Hive security using the basic authorization scheme, which regulates access to Hive metadata on the client side. Starting with Hive release 0.10, additional security measures can be enabled to regulate access on the metastore side, as described in Metastore Server Security below.

Disclaimer

Hive authorization is not completely secure. The basic authorization scheme is intended primarily to prevent good users from accidentally doing bad things, but makes no promises about preventing malicious users from doing malicious things. Starting in Hive release 0.10, however, metastore-side security can be enabled to prevent malicious access to metadata in a metastore server configuration.

Prerequisites

In order to use Hive authorization, there are two parameters that should be set in hive-site.xml:

<property>
  <name>hive.security.authorization.enabled</name>
  <value>true</value>
  <description>enable or disable the hive client authorization</description>
</property>

<property>
  <name>hive.security.authorization.createtable.owner.grants</name>
  <value>ALL</value>
  <description>the privileges automatically granted to the owner whenever a table gets created. 
   An example like "select,drop" will grant select and drop privilege to the owner of the table</description>
</property>

Note that, by default, the hive.security.authorization.createtable.owner.grants are set to null, which would result in the creator of a table having no access to the table.

Users, Groups, and Roles

At the core of Hive's authorization system are users, groups, and roles. Roles allow administrators to give a name to a set of grants which can be easily reused. A role may be assigned to users, groups, and other roles. For example, consider a system with the following users and groups:

• <User>: <Groups>
• user_all_dbs: group_db1, group_db2
• user_db1: group_db1
• user_db2: group_db2

If we wanted to restrict each user to a specific set of databases, we could use roles to build the authorization mechanism. The administrator would create two roles, called role_db1 and role_db2. The role_db1 role would provide privileges just for the first database, and the role_db2 role would provide privileges just for the second database. The administrator could then grant the role_db1 role to group_db1, or explicitly for the users in the group, and do the same for role_db2 with the users of the second database. In order to allow users who need to see all databases to get their appropriate privileges, a third role could be created called role_all_dbs, which would be granted role_db1 and role_db2. When user_all_dbs is granted the role_all_dbs role, the user implicitly is granted all the privileges of role_db1 and role_db2.

Hive roles must be created manually before being used, unlike users and groups. Users and groups are managed by the hive.security.authenticator.manager. When a user connects to a Metastore Server and issues a query, the Metastore will determine the username of the connecting user, and the groups associated with that ushive.security.authorization.ername. That information is then used to determine if the user should have access to the metadata being requested, by comparing the required privileges of the Hive operation to the user privileges using the following rules:

• User privileges (Has the privilege been granted to the user)
• Group privileges (Does the user belong to any groups that the privilege has been granted to)
• Role privileges (Does the user or any of the groups that the user belongs to have a role that grants the privilege)

By default, the Metastore uses the HadoopDefaultAuthenticator for determing user -> group mappings, which determines authorization by using the Unix usernames and groups on the machine where the Metastore is running. To make this more clear, consider a scenario where a user foo is a member of group bar on the machine running the Hive CLI, and connects to a Metastore running on a separate server that also has a user named foo, but on the Metastore Server, foo is a member of group baz. When an operation is executed, the Metastore will determine foo to be in the group baz.

Taking this a step further, it is also possible for the groups that a user belongs to on the Metastore Server may differ from the groups that the same user belongs to, as determined by HDFS. This could be the case if Hive or HDFS are configured to use non-default user -> group mappers, or the Metastore and the Namenode both use the defaults, but the processes are running on different machines, and the user -> group mappings are not the same on each machine.

It is important to realize that Hive Metastore only controls authorization for metadata, and the underlying data is controlled by HDFS, so if permissions and privileges between the two systems are not in sync, users may have access to metadata, but not the physical data. If the user -> group mappings across the Metastore and Namenode are not in sync, as in the scenarios above, a user may have the privileges required to access a table according to the Metastore, but may not have permission to access the underlying files according to the Namenode. This could also happen due to administrator intervention, if permissions on the files were changed by hand, but Metastore grants had not been updated.

Names of Users and Roles

Role names are case insensitive. That is, “marketing” and “MarkEting” refer to same role.

User names are case sensitive. This is because, unlike role names, user names are not managed within Hive.

Creating/Dropping/Using Roles

Create/Drop Role

CREATE ROLE role_name

DROP ROLE role_name

Grant/Revoke Roles

GRANT ROLE role_name [, role_name] ...
TO principal_specification [, principal_specification] ...
[WITH ADMIN OPTION]

REVOKE [ADMIN OPTION FOR] ROLE role_name [, role_name] ...
FROM principal_specification [, principal_specification] ...

principal_specification:
    USER user
  | GROUP group
  | ROLE role

Viewing Granted Roles

SHOW ROLE GRANT principal_specification
 
principal_specification:
    USER user
  | GROUP group
  | ROLE role

Privileges

The following privileges are supported in Hive:

• ALL - Gives users all privileges
• ALTER - Allows users to modify the metadata of an object
• UPDATE - Allows users to modify the physical data of an object
• CREATE - Allows users to create objects. For a database, this means users can create tables, and for a table, this means users can create partitions
• DROP - Allows users to drop objects
• INDEX - Allows users to create indexes on an object (Note: this is not currently implemented)
• LOCK - Allows users to lock or unlock tables when concurrency is enabled
• SELECT - Allows users to access data for objects
• SHOW_DATABASE - Allows users to view available databases

Grant/Revoke Privileges

GRANT
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    [ON object_type]
    TO principal_specification [, principal_specification] ...
    [WITH GRANT OPTION]

REVOKE [GRANT OPTION FOR]
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    [ON object_type priv_level]
    FROM principal_specification [, principal_specification] ...

REVOKE ALL PRIVILEGES, GRANT OPTION
    FROM user [, user] ...

priv_type:
    ALL | ALTER | UPDATE | CREATE | DROP
  | INDEX | LOCK | SELECT | SHOW_DATABASE 
 
object_type:
    TABLE
  | DATABASE

priv_level:
    db_name
  | tbl_name
 
principal_specification:
    USER user
  | GROUP group
  | ROLE role

Viewing Granted Privileges

SHOW GRANT principal_specification
[ON object_type priv_level [(column_list)]]
 
principal_specification:
    USER user
  | GROUP group
  | ROLE role
 
object_type:
    TABLE
  | DATABASE

priv_level:
    db_name
  | tbl_name

Hive Operations and Required Privileges

As of the release of Hive 0.7, only these operations require permissions, according to org.apache.hadoop.hive.ql.plan.HiveOperation:

Metastore Server Security

This section describes the metastore server security feature added to Hive in release 0.10. This feature was introduced previously in HCatalog (see Storage Based Authorization).

The Need for Metastore Server Security

When multiple clients access the same metastore in a backing database, such as MySQL, the database connection credentials may be visible in the hive-site.xml configuration file. A malicious or incompetent user could cause serious damage to metadata even though the underlying data is protected by HDFS access controls.

Also, when a Hive metastore server uses Thrift to communicate with clients and has a backing database for metadata storage and persistence, the authentication and authorization done on the client side cannot guarantee security on the metastore side.

To provide security for metadata, release 0.10 adds authorization capability to the metastore. (See HIVE-3705.) 

Storage Based Authorization

When metastore server security is configured to use Storage Based Authorization, it uses the file system permissions for folders corresponding to the different metadata objects as the source of truth for the authorization policy. Use of Storage Based Authorization in metastore is recommended.

See details in the Storage Based Authorization document.

Configuration Parameters for Metastore Security

To enable Hive metastore server security, set these parameters in hive-site.xml:

• hive.metastore.pre.event.listeners
  Set to org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener.
  This turns on metastore-side security.

• hive.security.metastore.authorization.manager
  Set to org.apache.hadoop.hive.ql.security.authorization.DefaultHiveMetastoreAuthorizationProvider.
  This tells Hive which metastore-side authorization provider to use. The default setting uses DefaultHiveMetastoreAuthorizationProvider, which implements the standard Hive grant/revoke model. To use an HDFS permission-based model (recommended) to do your authorization, you can use org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider instead.

  Version

  The StorageBasedAuthorizationProvider was introduced in Hive 0.10.0, running on the metastore side only (HIVE-3705). Starting in Hive 0.12.0 it also runs on the client side (HIVE-5048 and HIVE-5402).

• hive.security.metastore.authenticator.manager
  Set to org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator.

The snippet below shows the keys as they are in a default state in hive-site.xml (metastore-side security set up to use the default authorization/authentication, but disabled). Please edit in information as above to get the desired authorization behaviour:

<property>
  <name>hive.security.metastore.authorization.manager</name>
  <value>org.apache.hadoop.hive.ql.security.authorization.DefaultHiveMetastoreAuthorizationProvider</value>
  <description>authorization manager class name to be used in the metastore for authorization.
  The user defined authorization class should implement interface
  org.apache.hadoop.hive.ql.security.authorization.HiveMetastoreAuthorizationProvider.
  </description>
 </property>

<property>
  <name>hive.security.metastore.authenticator.manager</name>
  <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value>
  <description>authenticator manager class name to be used in the metastore for authentication.
  The user defined authenticator should implement interface 
  org.apache.hadoop.hive.ql.security.HiveAuthenticationProvider.
  </description>
</property>

<property>
  <name>hive.metastore.pre.event.listeners</name>
  <value> </value>
  <description>pre-event listener classes to be loaded on the metastore side to run code
  whenever databases, tables, and partitions are created, altered, or dropped.
  Set to org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener
  if metastore-side authorization is desired.
  </description>
</property>

SQL Standards Based Authorization

Hive release 0.13.0 introduced authorization based on SQL standards.  See SQL Standard Based Hive Authorization for details.