You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

– This document is a work in progress.

Overview

Templeton provides a REST-like web API for HCatalog and related Hadoop components. Developers can make HTTP requests to the Templeton web server to execute HCatalog DDL commands. With the REST APIs in place for HCatalog DDL commands, it is desirable to have a JAVA APIs in HCAT which can help end users to execute DDL commands without using CLI.

Design

New Classes:

HCatClient

The HCatClient is an interface containing all the APIs permitted HCatalog DDL commands.

 package org.apache.hcatalog.api;

import java.util.List;

import org.apache.hcatalog.common.HCatException;
import org.apache.hcatalog.mapreduce.HCatDatabaseInfo;
import org.apache.hcatalog.mapreduce.HCatPartitionInfo;
import org.apache.hcatalog.mapreduce.HCatTableInfo;

/**
 * The Interface HCatClient containing APIs for HCatalog DDL commands.
 */
public interface HCatClient {

    /**
     * Gets the database like.
     *
     * @param regex The regular expression. Providing "*" would retrieve all the names
     *              of the databases.
     * @return The list of all the database names.
     * @throws HCatException
     */
    public List<String> getDatabaseLike(String regex) throws HCatException;

    /**
     * Gets the database.
     *
     * @param dbName The name of the database.
     * @return An instance of HCatDatabaseInfo.
     * @throws HCatException
     */
    public HCatDatabaseInfo getDatabase(String dbName) throws HCatException;

    /**
     * Creates the database.
     *
     * @param dbInfo An instance of HCatCreateDBDesc.
     * @return true, if successful
     * @throws HCatException
     */
    public boolean createDatabase(HCatCreateDBDesc dbInfo)
            throws HCatException;

    /**
     * Deletes a database.
     *
     * @param dbName The name of the database to delete.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param mode This is set to either "restrict" or "cascade". Restrict will
     *             remove the schema if all the tables are empty. Cascade removes
     *             everything including data and definitions.
     * @param userGroup The user group to use
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deleteDatabase(String dbName, boolean ifExists, String mode,
            String userGroup, String permissions) throws HCatException;

    /**
     * Gets the tables like a pattern specified.
     *
     * @param dbName The name of the database.
     * @param regex The regular expression. Providing "*" would retrieve all the names
     *              of  the table.
     * @return A list of all table names matching the specified pattern.
     * @throws HCatException
     */
    public List<String> getTablesLike(String dbName, String regex)
            throws HCatException;

    /**
     * Gets the table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @return An instance of HCatTableInfo.
     * @throws HCatException
     */
    public HCatTableInfo getTable(String dbName, String tableName)
            throws HCatException;

    /**
     * Creates the table.
     *
     * @param createTableDesc An instance of HCatCreateTableDesc class.
     * @return true, if successful.
     * @throws HCatException the h cat exception
     */
    public boolean createTable(HCatCreateTableDesc createTableDesc)
            throws HCatException;

    /**
     * Creates the table like an existing table.
     *
     * @param dbName The name of the database.
     * @param existingTblName The name of the existing table.
     * @param newTableName The name of the new table.
     * @param ifExists the if exists
     * @param isExternal Set to "true", if table has be created at a different
     *                   location other than default.
     * @param location The location for the table.
     * @return true, if successful
     * @throws HCatException
     */
    public boolean createTableLike(String dbName, String existingTblName,
            String newTableName, boolean ifExists, boolean isExternal,
            String location) throws HCatException;

    /**
     * Delete a table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deleteTable(String dbName, String tableName,
            boolean ifExists, String userGroup, String permissions)
            throws HCatException;

    /**
     * Renames a table.
     *
     * @param dbName The name of the database.
     * @param oldName The name of the table to be renamed.
     * @param newName The new name of the table.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean renameTable(String dbName, String oldName, String newName,
            String userGroup, String permissions) throws HCatException;

    /**
     * Gets all the partitions.
     *
     * @param dbName The name of the database.
     * @param tblName The name of the table.
     * @return A list of partition names.
     * @throws HCatException the h cat exception
     */
    public List<HCatPartitionInfo> getPartitions(String dbName, String tblName)
            throws HCatException;

    /**
     * Gets the partition.
     *
     * @param dbName The database name.
     * @param tableName The table name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @return An instance of HCatPartitionInfo.
     * @throws HCatException
     */
    public HCatPartitionInfo getPartition(String dbName, String tableName,
            String partitionName) throws HCatException;

    /**
     * Adds the partition.
     *
     * @param partInfo An instance of HCatAddPartitionDesc.
     * @return true, if successful
     * @throws HCatException the h cat exception
     */
    public boolean addPartition(HCatAddPartitionDesc partInfo) throws HCatException;

    /**
     * Deletes partition.
     *
     * @param dbName The database name.
     * @param tableName The table name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @param ifExists Hive returns an error if the partition specified does not exist, unless ifExists is set to true.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deletePartition(String dbName, String tableName,
            String partitionName, boolean ifExists, String userGroup,
            String permissions) throws HCatException;

    /**
     * List partitions by filter.
     *
     * @param dbName The database name.
     * @param tblName The table name.
     * @param filter The filter string,
     *    for example "part1 = \"p1_abc\" and part2 <= "\p2_test\"". Filtering can
     *    be done only on string partition keys.
     * @return list of partitions
     * @throws HCatException the h cat exception
     */
    public List<HCatPartitionInfo> listPartitionsByFilter(String dbName, String tblName,
            String filter) throws HCatException;

}
HCatTempletonClient

This class implements HCatClient interface.

HCatTempletonDriver

This class implements Hive's CommandProcessorResponse interface.

public interface CommandProcessor {
  public void init();

  public CommandProcessorResponse run(String command) throws CommandNeedRetryException;
}

The "run" method will consume the curl command as an input parameter and return the response.

HCatCommandDesc

This is an abstract class that helps in validating user input, building valid command descriptors and queries.

/**
 * The Class HCatCommandDesc contains methods which help in validating,
 * building command descriptors and queries.
 */
public abstract class HCatCommandDesc{

    public abstract void validateCommandDesc() throws HCatException;
    abstract String buildQuery() throws HCatException;
    abstract boolean isValidationComplete();

}
HCatCreateTableDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptor and validate it for the "create table" command.

HCatCreateDBDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptos and validate it for the "create database" command.

HCatAddPartitionDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptos and validate it for the "add partition" command.

HCatDBInfo

Modification to Existing Classes:

HCatTableInfo
PartInfo

Usage

Concerns

  • No labels