Metastore 3.0 Administration

Version Note

This document applies only to the Metastore in Hive 3.0 and later releases.  For Hive 0.x, 1.x, and 2.x releases please see the Metastore Administration document.

Introduction

The definition of Hive objects such as databases, table,  and functions are stored in the Metastore.  Depending on how the system is configured, statistics and authorization data may also be stored there.  Hive, and other execution engines, can then use this data at runtime to determine how to efficiently execute user queries.  

The Metastore persists the object definitions to a relational database (RDBMS) via DataNucleus, a Java JDO based Object Relational Mapping (ORM) layer. See XXX below for a list of supported RDBMSs that can be used.

The Metastore can be configured to embed the Derby RDBMS or connect to a external RDBMS.  It can be embedded entirely in a user process or run as a service for other processes to connect to.  Each of these options will be discussed in turn below.

Changes From Hive 2.x to Hive 3.0

Beginning in Hive 3.0, the Metastore can be run without the of Hive being installed.  It is provided as a separate release in order to allow non-Hive systems to easily integrate with it.  (It is, however, still included in the Hive release for convenience.)  Making the Metastore a standalone service involved changing a number of configuration variable names and tool names.  All of the old configuration variables and tools still work for previously existing values and functions in order to maximize backwards compatibility.  This document will cover both the old and new names.  As new functionality is added it will only be added to the new names.

For details on using the Metastore without HIve, see XXX below.

General Configuration

The following configuration values apply to the Metastore regardless of how it is being run.

metastore.warehouse.dir

RDBMS

Option 1: Embedding Derby

Option 2: External RDBMS

Supported RDBMSs

Installing, Upgrading, and Checking Metastore Tables in the RDBMS

Running the Metastore

Embedding the Metastore in Your Process

Security Considerations

Metastore Server

Starting and Stopping the Service

High Availability

Securing the Service

Running the Metastore Without Hive