1. Is the partition key stored together with data?
No. Partition key becomes the name of the folder, it does not store along with data.

2. When I load data from HCat, is partition key part of output?
Yes. Partition key will be added to the end of the output.

3. When I store data, what if the schema I specified does not match the table definition?
HCatalog will throw exception. This happens when:

  • Column name does not match
  • Column type does not match
  • Column has different order
  • No labels

1 Comment

  1. Bian Zhouyong

    hcatalog0.6.0 has some problem on building with ant:
    if we change mvn.hadoop.profile=hadoop23,there will a error on building:
    ant -Dhcatalog.version=0.6.0 -Dforrest.home=/opt/forrest package

    javac /opt/tar/hcatalog-trunk/shims/src/23/java/org/apache/hcatalog/shims/HCatHadoopShims23.java:34: package org.apache.hadoop.mapreduce.task does not exist
    javac import org.apache.hadoop.mapreduce.task.JobContextImpl;

    the required jar file has been put in a local maven repo, and the dependency configuration is already in pom.xml:
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-archives</artifactId>
    <version>2.0.2-alpha</version>
    <scope>compile</scope>
    </dependency>
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.0.2-alpha</version>
    <scope>compile</scope>
    </dependency>
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>2.0.2-alpha</version>
    <scope>compile</scope>
    </dependency>
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-core</artifactId>
    <version>2.0.2-alpha</version>
    <scope>compile</scope>
    </dependency>
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-jobclient</artifactId>
    <version>2.0.2-alpha</version>
    <scope>compile</scope>
    </dependency>

    is there any mistake?