HCatalog includes two forms of testing, unit and end-to-end (e2e), or feature, testing.
Unit Tests
Run all unit tests
To run all unit tests, run the test script in the top level HCatalog directory.
./build-support/scripts/test.sh
This will run a series of unit tests that currently takes tens of minutes. If all the tests pass, you will see BUILD SUCCESSFUL
. If any fail, you can look in build/test/logs
for text files with output from each test. This script automates running the ant build/tests, ensuring everyone runs tests the same way; additionally, as our test suite evolves this provides a single place to make changes that everyone will run (including our CI job).
Run a specific unit test
Care has been taken to write many tests such that they run inside an IDE, however, running a specific test from the command-line is also supported.
ant clean test -Dtestcase=MyTest
Run tests with coverage
HCatalog is integrated with clover, which can be used to measure code coverage of the unit tests. First, you will need to obtain clover:
- Download clover from Atlassian. You want Clover for Ant 2.6.3. The license from Apache doesn't work with the latest 3.x versions of Clover.
- Unzip clover somewhere on the machine where you will be doing the tests. We'll call this location
$CLOVER_HOME
- Download the license file from Apache. You must be an Apache committer to access this file.
- Place the license file in
$CLOVER_HOME/lib
Now, to run the unit tests with clover:
ant clean ant -Dclover.home=<clover_home> -Drun.clover=true clover jar test ant -Dclover.home=<clover_home> -Drun.clover=true generate-clover-reports ant -Dclover.home=<clover_home> -Drun.clover=true generate-pdf-clover-reports
The detailed report is now available as a collection of HTML files under build/test/clover/reports
A summary report is now available in build/test/clover/pdf/reports/clover_coverage.pdf
End-to-end/Feature tests
To run the e2e tests, you need access to a Hadoop cluster with Pig, Hive, HCatalog, and HBase installed. On this cluster you need to assure the following:
- A directory
/user/hcat
exists and is owned by the user you will be running the tests as - A directory
/tmp
exists and has permissions of 777.
You also need to install the Perl module IPC::Run on the machine you will be running the tests from cpan install IPC::Run
You need to set the environment variable JAVA_HOME
to point to where Java is on your machine.
Finally, you need access to a MySQL database. This database holds the expected results of the tests. You can install MySQL on the same machine you are running the tests on. You will need to add a user hcattest
with a password of hcattest
and a database hcattestdb
. Make sure to grant permissions for user hcattest
to access hcattestdb
from localhost
(or your actual hostname if the MySQL db is on a separate machine).
Running e2e Tests with Tarball Installation
These instructions walk you through how to run the tests if you have installed Hadoop and related components via tarballs provided by each project.
In the following, you will need to know these values:
HADOOP_HOME
- where you installed HadoopHIVE_HOME
- where you installed HiveHCAT_HOME
- where you installed HCatalogPIG_HOME
- where you installed PigHBASE_HOME
- where you installed HBase
Before you run the tests, you must build the HCatalog jar. If you not already done so, you can do this by running ant
The first time you run the tests, you will need to deploy the data:
ant -Dhive.home=$HIVE_HOME -Dhcat.home=$HCAT_HOME -Dpig.home=$PIG_HOME -Dhadoop.home=$HADOOP_HOME -Dhbase.home=$HBASE_HOME test-e2e-deploy
Once this is done, you can run the tests any number of times by doing:
ant -Dhive.home=$HIVE_HOME -Dhcat.home=$HCAT_HOME -Dpig.home=$PIG_HOME -Dhadoop.home=$HADOOP_HOME -Dhbase.home=$HBASE_HOME test-e2e
Running e2e Tests with rpm Installation
These instructions walk you through how to run the tests if you have installed Hadoop and related components via BigTop rpm packages:
Before you run the tests, you must build the HCatalog jar. If you not already done so, you can do this by running ant
The first time you run the tests, you will need to deploy the data:
ant -Dhive.home=/usr/lib/hive/ \ -Dhcat.home=/usr/lib/hcatalog/ \ -Dpig.home=/usr/lib/pig/ \ -Dhadoop.home=/usr/lib/hadoop/ \ -Dhbase.home=/usr/lib/hbase/ \ test-e2e-deploy
Once this is done, you can run the tests any number of times by doing:
ant -Dharness.conf=`pwd`/src/test/e2e/hcatalog/conf/rpm.conf test-e2e