You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 14 Next »

Hive Developer FAQ

Mavenization is pending

Until HIVE-5610 is closed the maven commands below will not work. Afterwards the ant commands should be removed.

Building

Maven settings

You might have to set the following maven options on certain systems to get build working. Check out the suggested value for MAVEN_OPTS under the testing section.

How to build all source?

MVN:

mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

How do I import into eclipse?

Build and generate eclipse files:

$ mkdir workspace
$ cd workspace
$ mvn clean install -DskipTests
$ mvn eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs -Declipse.workspace==/path/to/workspace
$ cd itests
$ mvn clean install -DskipTests
$ mvn eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs
$ cd ..
$ find . -name '.classpath' | xargs perl -i -pe 's@ <classpathentry kind="src" path="/hive-shims"/>@@g'

In eclipse define M2_REPO in Preferences -> Java -> Build Path -> Classpath Variables to either:

Mac Example:

/Users/$USER/.m2/repository

Linux Example:

/home/$USER/.m2/repository

Then import the workspaces. If you an error about "restricted use of Signal" for beeline and cli follow the following instructions.

How to generate protbuf code?

MVN:

cd ql
mvn clean install -DskipTests -Pprotobuf -Phadoop-1

How to generate thrift code?

MVN:

mvn clean install -Pthriftif -Phadoop-1 -DskipTests -Dthrift.home=/usr/local

How to compile ODBC?

MVN:

cd odbc
mvn compile -Podbc -Dthrift.home=/usr/local -Dboost.home=/usr/local

How do I publish hive artifacts to my local maven repository?

ant package
ant -Dmvn.publish.repo=local maven-build
ant -Dmvn.publish.repo=local maven-publish

MVN:

mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

Testing

How do I run all of the unit tests?

Make sure that your JAVA_HOME is appropriately set (some tests need this), and set ANT_OPTS to increase the size allocated to the Permanent Generation as per the following:

export ANT_OPTS="-XX:MaxPermSize=512m"
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=256M"

Then, for a clean build, run

ant clean package test

MVN:

mvn test
cd itests 
mvn test

Note that running ant test will not work; ant package does some setup work that is required for the testcases to run successfully.

MVN: Note that you need to have previously built and installed the jars:

mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

How do update the output of a CliDriver testcase?

ant test -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true

MVN:

cd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true

As of Hive 0.11.0+ you can cut this time in half by specifying that only the ql module needs to rebuild

ant test -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true

How do I run the clientpositive/clientnegative unit tests?

All of the below require that you have previously run ant package.

To run clientpositive tests

ant -Dtestcase=TestCliDriver test

MVN:

cd itests/qtest
mvn test -Dtest=TestCliDriver

To run a single clientnegative test alter1.q

ant -Dtestcase=TestNegativeCliDriver -Dqfile=alter1.q test

MVN:

cd itests/qtest
mvn test -Dtest=TestNegativeCliDriver -Dqfile=alter1.q

To run all of the clientpositive tests that match a regex, e.g. the partition_wise_fileformat tests

ant -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test

MVN:

cd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile_regex=partition_wise_fileformat.*

To run a single contrib test alter1.q and overwrite the result file

ant -Dtestcase=TestContribCliDriver -Dqfile=alter1.q -Doverwrite=true test

MVN:

cd itests/qtest
mvn test -Dtest=TestContribCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true

To run a single test groupby1.q and output detailed information during execution

ant -Dtestcase=TestCliDriver -Dqfile=groupby1.q -Dtest.silent=false test

As of Hive 0.11.0+ you can cut down the total build time by specifying that only the ql module needs to rebuild. e.g. run all the partition_wise_fileformat tests

ant -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test
  • No labels