Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

There are presently three ways to issue HCatalog DDL commands:

  1. Command line interface
  2. Templeton Upcoming REST APIs
  3. HiveMetaStore Client

...

The HCatClient is an abstract class containing all the APIs permitted HCatalog DDL commands. The implementation class will be provided as a configuration property, which will be used by the
"create" method. In this way, the implementation details will be masked to the users.

Code Block

/**
 * The abstract class HCatClient containing APIs for HCatalog DDL commands.
 */
public abstract class HCatClient {

    /**
     * Creates an instance of HCatClient.
     *
     * @param conf An instance of configuration.
     * @return An instance of HCatClient.
     */
    public static HCatClient create(Configuration conf){
        // Obtain details of implementation class and return an instance.return null;
    }

    /**
     * GetsGet theall databaseexisting like.
databases that match the  *given
     * @param regex The regular expression. Providing "*" would retrieve all the namespattern. The matching occurs as per Java regular expressions
     *
     * @param databasePattern
     *       of the databases.   java re pattern
     * @return The list of all the database names.
     * @throws HCatException
     */
    public abstract List<String> getDatabaseslistDatabasesByPattern(String regexpattern) throws HCatException;

    /**
     * Gets the database.
     *
     * @param dbName The name of the database.
     * @return An instance of HCatDatabaseInfo.
     * @throws HCatException
     */
    public abstract HCatDatabase getDatabase(String dbName) throws HCatException;

    /**
     * Creates the database.
     *
     * @param dbInfo An instance of HCatCreateDBDesc.
     * @return true, if successful@throws HCatException
     * @throws HCatException/
     */
    public abstract booleanvoid createDatabase(HCatCreateDBDesc dbInfo)
            throws HCatException;

    /**
     * DeletesDrops a database.
     *
     * @param dbName The name of the database to delete.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param mode This is set to either "restrict" or "cascade". Restrict will
     *             remove the schema if all the tables are empty. Cascade removes
     *             everything including data and definitions.
     * @param@throws userGroupHCatException
 The user group to use*/
    public abstract void * @param permissions The permissions string to use. The format is "rwxrw-r-x".dropDatabase(String dbName, boolean ifExists, String mode) throws HCatException;

    /**
     * @return true, if successful
     * @throws HCatExceptionReturns all existing tables from the specified database which match the given
     */
 pattern. The matching publicoccurs abstractas booleanper deleteDatabase(String dbName, boolean ifExists, String mode,
    Java regular expressions.
     * @param dbName
     * @param tablePattern
 String userGroup, String permissions) throws HCatException;

    /*** @return list of table names
     * Gets@throws theHCatException
 tables like a pattern specified.*/
    public *
abstract List<String> listTablesByPattern(String dbName, String *tablePattern)
 @param dbName The name of the database.
     * @param regex The regular expression. Providing "*" would retrieve all the namesthrows HCatException;

    /**
     * Gets the table.
     *
     * @param dbName The name     of  the tabledatabase.
     * @param @returntableName AThe listname of allthe table names matching the specified pattern.
     * @return An instance of HCatTableInfo.
     * @throws HCatException
     */
    public abstract List<String>HCatTable getTablesgetTable(String dbName, String regextableName)
            throws HCatException;

    /**
     * GetsCreates the table.
     *
     * @param dbNamecreateTableDesc TheAn nameinstance of theHCatCreateTableDesc databaseclass.
     * @param tableName The name of@throws HCatException the table.
h     * @return An instance of HCatTableInfo.cat exception
     * @throws HCatException
     */
    public abstract HCatTablevoid getTablecreateTable(String dbName, String tableNameHCatCreateTableDesc createTableDesc)
            throws HCatException;

    /**
     * Creates the table.
 like an existing table.
     *
     * @param createTableDescdbName AnThe instancename of HCatCreateTableDescthe classdatabase.
     * @return true, if successful@param existingTblName The name of the existing table.
     * @throws HCatException@param newTableName The name of the h cat exceptionnew table.
     */
 @param ifExists the if publicexists
 abstract boolean createTable(HCatCreateTableDesc createTableDesc)
 * @param isExternal Set to "true", if table has be created throws HCatException;

    /**at a different
     * Creates the table like an existing table.
     *
     * @param dbNamelocation Theother name of the databasethan default.
     * @param existingTblNamelocation The namelocation offor the existing table.
     * @param@throws newTableNameHCatException
 The name of the new table. */
    public *abstract @paramvoid ifExists the if existscreateTableLike(String dbName, String existingTblName,
     * @param isExternal Set to "true", if tableString hasnewTableName, beboolean createdifExists, at a differentboolean isExternal,
     *       String location) throws HCatException;

     /**
    location other* thanDrop default.table.
     *
     * @param locationdbName The locationname forof the tabledatabase.
     * @return true, if successful@param tableName The name of the table.
     * @throws@param HCatException
ifExists Hive returns an  */
    public abstract boolean createTableLike(String dbName, String existingTblName,
error if the database specified does not exist,
     *            String newTableName, boolean ifExists, boolean isExternal,
unless ifExists is set to true.
     *  String location) throws HCatException;

@throws HCatException
     */**
     * Delete a table.public abstract void dropTable(String dbName, String tableName,
     *
     * @param dbNameboolean TheifExists) name of the database.throws HCatException;

    /**
 * @param tableName The name* ofRenames thea table.
     *
 @param ifExists Hive returns an* error@param ifdbName theThe databasename specifiedof does not exist,the database.
     * @param oldName The name of the table to be renamed.
     * @param unlessnewName The ifExistsnew isname setof tothe truetable.
     * @param@throws userGroupHCatException
 The user group to use.*/
    public *abstract @paramvoid permissions The permissions string to use. The format is "rwxrw-r-x".renameTable(String dbName, String oldName, String newName) throws HCatException;

     /**
 @return true, if successful
 * Gets all  * @throws HCatExceptionthe partitions.
     */
    public abstract* boolean@param deleteTable(String dbName, StringThe tableName,
name of the database.
     * @param tblName The booleanname ifExists,of String userGroup, String permissions)the table.
     * @return A list of partition names.
 throws HCatException;

    /**
 @throws HCatException the h * Renames a table.cat exception
     */
    public abstract *List<HCatPartition> @paramgetPartitions(String dbName, The name of the database.String tblName)
     * @param oldName The name of the table to be renamed.throws HCatException;

     /**
 @param newName The new name* ofGets the table.partition.
     *
     * @param userGroupdbName The user group to usedatabase name.
     * @param permissionstableName The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successfultable name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @return An instance of HCatPartitionInfo.
     * @throws HCatException
     */
    public abstract booleanHCatPartition renameTablegetPartition(String dbName, String oldNametableName, String newName,
            String userGroup, String permissionspartitionName) throws HCatException;

    /**
     * Gets allAdds the partitionspartition.
     *
     * @param dbNamepartInfo TheAn nameinstance of the databaseHCatAddPartitionDesc.
     * @param@throws tblNameHCatException Thethe nameh of the table.cat exception
     * @return A list of partition names./
    public abstract void addPartition(HCatAddPartitionDesc partInfo) throws HCatException;

    /**
 * @throws HCatException the h* catDrops exceptionpartition.
     */
    public abstract List<HCatPartition> getPartitions(String* @param dbName, String tblName)
            throws HCatException;

    /**
     * Gets the partition.
     *
     * @param dbName The The database name.
     * @param tableName The table name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @return@param An instance of HCatPartitionInfoifExists Hive returns an error if the partition specified does not exist, unless ifExists is set to true.
     * @throws HCatException
     */
    public abstract HCatPartitionvoid getPartitiondropPartition(String dbName, String tableName,
            String partitionName, boolean ifExists) throws HCatException;

    /**
     * List Addspartitions theby partitionfilter.
     *
     * @param partInfodbName AnThe instancedatabase of HCatAddPartitionDescname.
     * @return@param true, if successfultblName The table name.
     * @throws@param HCatExceptionfilter theThe h cat exceptionfilter string,
     */
    publicfor abstractexample boolean addPartition(HCatAddPartitionDesc partInfo) throws HCatException;

    /**"part1 = \"p1_abc\" and part2 <= "\p2_test\"". Filtering can
     *  Deletes partition  be done only on string partition keys.
     * @return list of partitions
     * @param@throws HCatException dbNamethe Theh databasecat name.exception
     */
 @param tableName The tablepublic name.
abstract List<HCatPartition> listPartitionsByFilter(String dbName, String tblName,
 * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @param ifExists Hive returns an error if the partition specified does not exist, unless ifExists is set to true.  String filter) throws HCatException;

    /**
     * Mark partition for event.
     *
     * @param userGroupdbName The user group to usedatabase name.
     * @param permissionstblName The permissions string to use. The format is "rwxrw-r-x".table name.
     * @param partKVs the part k vs
     * @param @returneventType true,the ifevent successfultype
     * @throws HCatException the h cat exception
     */
    public abstract booleanvoid deletePartitionmarkPartitionForEvent(String dbName, String tableNametblName,
            String partitionNameMap<String, booleanString> ifExistspartKVs, StringPartitionEventType userGroup,eventType)
            String permissions) throws HCatException;

    /**
     * List partitions by filter Checks if is partition marked for event.
     *
     * @param dbName Thethe databasedb name.
     * @param tblName Thethe tabletbl name.
     * @param partKVs filterthe Thepart filterk string,vs
     * @param eventType the forevent exampletype
 "part1 = \"p1_abc\" and part2* <= "\p2_test\"". Filtering can
     *    be done only on string partition keys.
     * @return list of partitions@return true, if is partition marked for event
     * @throws HCatException the h cat exception
     */
    public abstract List<HCatPartition>boolean listPartitionsByFilterisPartitionMarkedForEvent(String dbName, String tblName,
            StringMap<String, filter)String> throws HCatException;

partKVs, PartitionEventType eventType)
    /**
     * Mark partition for event.throws HCatException;

     *
     * @param dbName The database name./**
     * @paramGets tblNamethe Thedelegation table nametoken.
     * @param partKVsrenewerKerberosPrincipalName the partrenewer kerberos kprincipal vsname
     * @param eventType@return the eventdelegation typetoken
     * @throws HCatException the h cat exception
     */
    public abstract voidString markPartitionForEventgetDelegationToken(String dbName, String tblName,
     renewerKerberosPrincipalName) throws
       Map<String, String> partKVs, PartitionEventType eventType)
            throws HCatException;

    /**
     * ChecksRenew if is partition marked for eventdelegation token.
     *
     * @param dbNametokenStrForm the token dbstr nameform
     * @param@return tblName the tbl namelong
     * @param@throws partKVsHCatException the parth kcat vsexception
     * @param eventType the event type/
    public abstract long renewDelegationToken(String tokenStrForm) throws HCatException;

     /**
 @return true, if is partition* markedCancel fordelegation eventtoken.
     *
  @throws HCatException the h* cat@param exception
tokenStrForm the token str  */form
    public abstract* boolean@throws isPartitionMarkedForEvent(String dbName, String tblName,
            Map<String, String> partKVs, PartitionEventType eventType)
            throws HCatException;

    /**
     * Gets the delegation token.
     *
     * @param owner the owner
     * @param renewerKerberosPrincipalName the renewer kerberos principal name
     * @return the delegation token
     * @throws HCatException the h cat exception
     */
    public abstract String getDelegationToken(String owner, String renewerKerberosPrincipalName) throws
        HCatException;

    /**
     * Renew delegation token.
     *
     * @param tokenStrForm the token str form
     * @return the long
     * @throws HCatException the h cat exception
     */
    public abstract long renewDelegationToken(String tokenStrForm) throws HCatException;

    /**
     * Cancel delegation token.
     *
     * @param tokenStrForm the token str form
     * @throws HCatException the h cat exception
     */
    public abstract void cancelDelegationToken(String tokenStrForm) throws HCatException;

}
HCatCommandDesc

This is an abstract class that helps in validating user input and building command descriptors.

Code Block

abstract class HCatCommandDesc{

     String dbName;
     String userGroup;
     String permissions;

   protected static abstract class Builder {

        String innerUserGroup;
         String innerPerms;
         String innerDBName;

         protected Builder(String dbName){
            this.innerDBName = dbName;
        }

        public Builder setUserGroup(String user){
            this.innerUserGroup = user;
            return this;
        }

        /**
         * Sets the permissions.
         *
         * @param perms The permissions string to use. The format is "rwxrw-r-x".
         */
        protected Builder setPermissions(String perms){
            this.innerPerms = perms;
            return this;
        }

        protected abstract HCatCommandDesc build() throws HCatException;


    }

    protected HCatCommandDesc(String dbName, String userGroup, String permissions) {
    this.dbName = dbName;
    this.userGroup = userGroup;
    this.permissions = permissions;
}

    public String getDatabaseName() {
        return this.dbName;
    }

    public String getUserGroup(){
        return this.userGroup;
    }


    public String getPermissions(){
        return this.permissions;
    }


}HCatException the h cat exception
     */
    public abstract void cancelDelegationToken(String tokenStrForm) throws HCatException;

}   
HCatCreateTableDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptor and validate it for the "create table" command.
Image Modified

HCatCreateDBDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptors and validate it for the "create database" command.

Image Modified

HCatAddPartitionDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptors and validate it for the "add partition" command.

Image RemovedImage Added

HCatTable

This class encapsulates the table information returned the HCatClient implementation class and provides a uniform view to the user.

Image RemovedImage Added

HCatDatabase

This class encapsulates the database information returned the HCatClient implementation class and provides a uniform view to the user.

Image RemovedImage Added

HCatPartition

This class encapsulates the partition information returned the HCatClient implementation class and provides a uniform view to the user.

Image RemovedImage Added

Usage

Code Block
 Configuration config = new Configuration();
 config.add("hive-site.xml");
 HCatClient client = HCatClient.create(config);
 ArrayList<HCatFieldSchema> cols = new ArrayList<HCatFieldSchema>();
 cols.add(new HCatFieldSchema("col1", Type.INT, "comment1"));
 cols.add(new HCatFieldSchema("col2", Type.STRING, "comment2"));
HCatCreateTableDesc desc = new HCatCreateTableDesc.Builder("default","demo_table").setFileFormat("rcfile").setCols(cols).build();

 boolean success = client.createTable(desc);

...