Page History

...

...

In master, only Hadoop 2.x is supported, thus there is no need to specify a Maven profile for most build operations.

In master, MVN:

mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

...

Note that if you use the Hive git base directory as the Eclipse workspace, then it does not pick the right project names (for example, picks 'ant' instead of 'hive-ant'). Therefore it's recommended to have the workspace directory one up from the git directory. For example workspaces/hive-workspace/hive where hive-workspace is the Eclipse workspace and hive is the git base directory.

How to generate tarball?

MVN:

mvn clean package -DskipTests -Pdist

...

How to generate protobuf code?

MVN:

cd ql
mvn clean install -DskipTests -Pprotobuf

How to generate Thrift code?

MVN:

mvn clean install -Pthriftif -DskipTests -Dthrift.home=/usr/local

How to run findbugs after a change?

mvn site -Pfindbugs 

How to run findbugs after a change?

mvn site -Pfindbugs 

Note: Note:  Available in Hive 1.1.0 onward (see HIVE-8327).

How to compile ODBC?

MVN:

cd odbc
mvn compile -Podbc -Dthrift.home=/usr/local -Dboost.home=/usr/local

How do I publish Hive artifacts to my local Maven repository?

ant package
ant -Dmvn.publish.repo=local maven-build
ant -Dmvn.publish.repo=local maven-publish

MVN:

mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

Testing

For general information, see Unit Tests and Debugging in the Developer Guide.

...

mvn clean install -DskipTests 
cd itests 
mvn clean install -DskipTests 

How do I run all of the unit tests except for a certain few tests?

Similar to running all tests, but define test.excludes.additional to specify a test/pattern to exclude from the test run. For example the following will run all tests except for the CliDriver tests:

Then, for a clean build, run

export ANT_OPTS="-XX:MaxPermSize=512m"

ant clean package test

Note that running ant test will not work; ant package does some setup work that is required for the testcases to run successfully.

How do I run all of the unit tests except for a certain few tests?

Similar to running all tests, but define test.excludes.additional to specify a test/pattern to exclude from the test run. For example the following will run all tests except for the CliDriver tests:

cd itests 
mvn test -Dtest.excludes.additional='**/Test*CliDriver.java' 

How do I run the clientpositive/clientnegative unit tests? 

All of the below require that you have previously run ant package.

To run clientpositive tests

cd itests/qtest
mvn test -Dtest=TestCliDriver 

To run a single clientnegative test alter1.q

cd itests/qtest
mvn test -Dtest=TestNegativeCliDriver -Dqfile=alter1.q 

To run all of the clientpositive tests that match a regex, for example the partition_wise_fileformat tests

cd itests/qtest

cd itests 
mvn test -Dtest.excludes.additional='**/Test*CliDriver.java' 

How do I run the clientpositive/clientnegative unit tests? 

All of the below require that you have previously run ant package.

To run clientpositive tests

ant -Dtestcase=TestCliDriver test

MVN:

cd itests/qtest
mvn test -Dtest=TestCliDriver 

To run a single clientnegative test alter1.q

ant -Dtestcase=TestNegativeCliDriver -Dqfile=alter1.q test

MVN:

cd itests/qtest=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* 

# Alternatively, you can specify comma separated list with "-Dqfile" argument
mvn test -Dtest=TestNegativeCliDriverTestMiniLlapLocalCliDriver -Dqfile=alter1'vectorization_0.q,vectorization_17.q,vectorization_8.q '

To run all of the clientpositive tests that match a regex, for example the partition_wise_fileformat tests

ant -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test

MVN:

a single contrib test alter1.q and overwrite the result file

cd itests/qtest
mvn test -Dtest=TestCliDriverTestContribCliDriver -Dqfile_regex=partition_wise_fileformat.* 

To run a single contrib test alter1.q and overwrite the result file

ant -Dtestcase=TestContribCliDriver -Dqfile=alter1.q -Doverwrite=true test

...

=alter1.q -Dtest.output.overwrite=true 

How do I remote debug a qtest?

cd itests/qtest
mvn test -Dtest=TestContribCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true 

To run a single test groupby1.q and output detailed information during execution

ant -DtestcaseDmaven.surefire.debug="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Xnoagent -Djava.compiler=NONE" test -Dtest=TestCliDriver -Dqfile=groupby1<test>.q -Dtest.silent=false test

As of Hive 0.11.0+ you can cut down the total build time by specifying that only the ql module needs to rebuild. For example, run all the partition_wise_fileformat tests

ant -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test

How do I modify How do I modify the init script when testing?

The option to skip the init script or supply a custom init script was added in Hive 2.0 (see HIVE-11538).

To skip initialization:

mvn test -Dtest=TestCliDriver -Phadoop-2 -Dqfile=test_to_run.q  -DinitScript=

To supply a custom script:

mvn test -Dtest=TestCliDriver -Phadoop-2 -Dtest.output.overwrite=true -Dqfile=test_to_run.q  -DinitScript=

To supply a custom

...

script

...

How do I update the output of a CliDriver testcase?

ant test -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true

MVN:

cd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile=alter1.qPhadoop-2 -Dtest.output.overwrite=true 

...

-Dqfile=test_to_run.q  -DinitScript=custom_script.sql

How do I update the output of a CliDriver testcase?

ant test -Dmodule=ql -Dtestcasecd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile=alter1.q -DoverwriteDtest.output.overwrite=true 

How do I update the results of many test cases?

...

• From the root of the source tree: find . -name hive.log

• /tmp/$USER/ (Linux) or $TMPDIR/$USER/ (MacOS)

See Hive Logging for details about log files, including alternative configurations.

How do I add a test case?

First, add the test case to the qfile test suite:

• hive.log

• /tmp/$USER/ (Linux) or $TMPDIR/$USER/ (MacOS)

See Hive Logging for details about log files, including alternative configurations.

How do I add a test case?

First, add the test case to the qfile test suite:

• Copy the test to a new file under ql/src/test/queries/clientpositive/<filename>.q (or /clientnegative if it is a negative test).
  • If the new test creates any table, view, function, etc., make sure that the name is unique across tests. For instance, name a table in the test file foo.q, foo_t1 instead of simply t1. This will help reduce flakiness in the test runs, since Jenkins will run tests and batches, and currently it does not restore to former state after running each of the q files.
  • If there is any interaction with file system, use unique folders for the test to avoid any collision with other tests
  Copy the test to a new file under ql/src/test/queries/clientpositive/<filename>.q (or /clientnegative if it is a negative test)
  • .
• Add the <filename.q> to itests/src/test/resources/testconfiguration.properties to the appropriate variable (ex. minimr.query.files).

...

• Run the test and generate the output file using the appropriate -Dtest (ex. TestMinimrCliDriver TestCliDriver; see itests/qtest/pom.xml for the names of other test suites):

  Code Block
  language text
  cd qtest mvn test -Dtest=TestMinimrCliDriverTestCliDriver -Dqfile=<filename>.q -Dtest.output.overwrite=true

  
  

...

At this point, attach the remote debugger as mentioned before to start debugging your queries.