Page History

NOTE: This Wiki is obsolete as of November 2016 and is retained for reference only.

Unit Testing

To run an individual Hive compatibility test:

...

To generate golden answer files based on Hive 0.12, you need to setup your development environment according to the "Other dependencies for developers" of this README. To generate golden answer files based on Hive 0.13.1, please follow the following steps.

1. Download Hive's 0.13.1 release and set HIVE_HOME (HIVE_DEV_HOME is not needed. See 

   Jira
   server ASF JIRA key SPARK-4119

    for details).

2. Set HADOOP_HOME.
3. Download all 0.13.1a jars from http://mvnrepository.com/artifact/org.spark-project.hive and replace corresponding jars in $HIVE_HOME/lib.
4. Download kryo 2.21.jar (Note: 2.22 jar does not work) and javolution 5.5.1 jar (http://mvnrepository.com/artifact/javolution/javolution/5.5.1) to $HIVE_HOME/lib.
5. You may not need this step. But, if a Hive query fails and you find that Hive tries to talk to HDFS or you find weird runtime NPEs, set the following in your test suite...

val testTempDir = Utils.createTempDir()
// We have to use kryo to let Hive correctly serialize some plans.
sql("set hive.plan.serialization.format=kryo")
// Explicitly set fs to local fs.
sql(s"set fs.default.name=file://$testTempDir/")
// Ask Hive to run jobs in-process as a single map and reduce task.
sql("set mapred.job.tracker=local")

After running the new test cases for the first time, golden answer files should be found under the sql/hive/src/test/resources/golden folder.