HCatalog includes two forms of testing, unit and end-to-end (e2e), or feature, testing.

Unit Tests

Run all unit tests

To run all unit tests, run the test script in the top level HCatalog directory.

./build-support/scripts/test.sh

This will run a series of unit tests that currently takes tens of minutes. If all the tests pass, you will see BUILD SUCCESSFUL. If any fail, you can look in build/test/logs for text files with output from each test. This script automates running the ant build/tests, ensuring everyone runs tests the same way; additionally, as our test suite evolves this provides a single place to make changes that everyone will run (including our CI job).

Run a specific unit test

Care has been taken to write many tests such that they run inside an IDE, however, running a specific test from the command-line is also supported.

ant clean test -Dtestcase=MyTest

Run tests with coverage

HCatalog is integrated with clover, which can be used to measure code coverage of the unit tests. First, you will need to obtain clover:

  1. Download clover from Atlassian. You want Clover for Ant 2.6.3. The license from Apache doesn't work with the latest 3.x versions of Clover.
  2. Unzip clover somewhere on the machine where you will be doing the tests. We'll call this location $CLOVER_HOME
  3. Download the license file from Apache. You must be an Apache committer to access this file.
  4. Place the license file in $CLOVER_HOME/lib

Now, to run the unit tests with clover:

ant clean
ant -Dclover.home=<clover_home> -Drun.clover=true clover jar test
ant -Dclover.home=<clover_home> -Drun.clover=true generate-clover-reports
ant -Dclover.home=<clover_home> -Drun.clover=true generate-pdf-clover-reports

The detailed report is now available as a collection of HTML files under build/test/clover/reports
A summary report is now available in build/test/clover/pdf/reports/clover_coverage.pdf

End-to-end/Feature tests

To run the e2e tests, you need access to a Hadoop cluster with Pig, Hive, HCatalog, and HBase installed. On this cluster you need to assure the following:

  1. A directory /user/hcat exists and is owned by the user you will be running the tests as
  2. A directory /tmp exists and has permissions of 777.

You also need to install the Perl module IPC::Run on the machine you will be running the tests from cpan install IPC::Run

You need to set the environment variable JAVA_HOME to point to where Java is on your machine.

Finally, you need access to a MySQL database. This database holds the expected results of the tests. You can install MySQL on the same machine you are running the tests on. You will need to add a user hcattest with a password of hcattest and a database hcattestdb. Make sure to grant permissions for user hcattest to access hcattestdb from localhost (or your actual hostname if the MySQL db is on a separate machine).

Running e2e Tests with Tarball Installation

These instructions walk you through how to run the tests if you have installed Hadoop and related components via tarballs provided by each project.
In the following, you will need to know these values:

  • HADOOP_HOME - where you installed Hadoop
  • HIVE_HOME - where you installed Hive
  • HCAT_HOME - where you installed HCatalog
  • PIG_HOME - where you installed Pig
  • HBASE_HOME - where you installed HBase

Before you run the tests, you must build the HCatalog jar. If you not already done so, you can do this by running ant

The first time you run the tests, you will need to deploy the data:

ant -Dhive.home=$HIVE_HOME -Dhcat.home=$HCAT_HOME -Dpig.home=$PIG_HOME -Dhadoop.home=$HADOOP_HOME -Dhbase.home=$HBASE_HOME test-e2e-deploy

Once this is done, you can run the tests any number of times by doing:

ant -Dhive.home=$HIVE_HOME -Dhcat.home=$HCAT_HOME -Dpig.home=$PIG_HOME -Dhadoop.home=$HADOOP_HOME -Dhbase.home=$HBASE_HOME test-e2e

Running e2e Tests with rpm Installation

These instructions walk you through how to run the tests if you have installed Hadoop and related components via BigTop rpm packages:

Before you run the tests, you must build the HCatalog jar. If you not already done so, you can do this by running ant

The first time you run the tests, you will need to deploy the data:

ant -Dhive.home=/usr/lib/hive/ \
    -Dhcat.home=/usr/lib/hcatalog/ \
    -Dpig.home=/usr/lib/pig/ \
    -Dhadoop.home=/usr/lib/hadoop/ \
    -Dhbase.home=/usr/lib/hbase/ \
    test-e2e-deploy

Once this is done, you can run the tests any number of times by doing:

ant -Dharness.conf=`pwd`/src/test/e2e/hcatalog/conf/rpm.conf test-e2e
  • No labels