Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Hive

...

Developer

...

FAQ

...

Table of Contents
Info
titleMaven

Hive is using Maven as its build tool. Versions prior to 0.13 were using Ant.

Developing

How do I move some files?

Post a patch for testing purposes which simply does add and deletes. SVN will not understand these patches are actually moves, therefore you should actually upload the following, in order so the last upload is the patch for testing purposes:

  1. A patch which has only the non-move changes for commit e.g. HIVE-XXX-for-commit.patch

...

  1. A

...

  1. script

...

  1. of

...

  1. of

...

  1. commands

...

  1. required

...

  1. to

...

  1. make

...

  1. the

...

  1. moves

...

  1. HIVE-XXX-moves.sh

...

  1. A

...

  1. patch

...

  1. for

...

  1. testing

...

  1. purposes

...

  1. HIVE-XXX.patch

...

The

...

script

...

should

...

be

...

a

...

set

...

of

...

svn

...

mv

...

commands

...

along

...

with

...

any

...

perl

...

commands

...

required

...

for

...

find/replace.

...

E.g.:

No Format


{format}
$ svn mv MyCLass.java MyClass.java
$ perl -i -pe 's@MyCLass@MyClass@gs<at:var at:name="MyCLass" />MyClass@g' MyClass.java
{format}

h2. Building

h3. How do I rerun precommit tests over the same patch?

Upload the exact same patch again to the JIRA.

h3. Maven settings
You might have to set the following maven options on certain systems to get build working. Check out the suggested value for MAVEN_OPTS 

Building

How do I rerun precommit tests over the same patch?

Upload the exact same patch again to the JIRA.

Maven settings

You might have to set the following maven options on certain systems to get build working. Check out the suggested value for MAVEN_OPTS under the testing section.

How to build all source?

MVN:

Code Block
under the testing section.

h3. How to build all source?

MVN:
{code}
mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests
{code}

h3. How do I import into eclipse?

Build and generate eclipse files (the conservative method):
{

How do I import into eclipse?

Build and generate eclipse files (the conservative method):

Code Block
code}
$ mkdir workspace
$ cd workspace
$ git clone https://github.com/apache/hive.git
$ cd hive
$ mvn clean install -DskipTests
$ mvn eclipse:clean
$ mvn eclipse:eclipse -DdownloadSources -DdownloadJavadocs
$ cd itests
$ mvn clean install -DskipTests
$ mvn eclipse:clean
$ mvn eclipse:eclipse -DdownloadSources -DdownloadJavadocs
{code}

In

...

eclipse

...

define

...

M2_REPO

...

in

...

Preferences

...

->

...

Java

...

->

...

Build

...

Path

...

->

...

Classpath

...

Variables

...

to

...

either:

...

Mac

...

Example:

{
Code Block
}
/Users/$USER/.m2/repository
{code}

Linux

...

Example:

{
Code Block
}
/home/$USER/.m2/repository
{code}

Windows

...

Example:

{
Code Block
}
C:/users/$USER/.m2/repository
{code}

Then

...

import

...

the

...

workspaces.

...

If

...

you

...

an

...

error

...

about

...

"restricted

...

use

...

of

...

Signal"

...

for

...

beeline

...

and

...

cli

...

follow

...

the

...

following

...

instructions.

Note that if you use the hive git base dir as eclipse workspace, then it does not pick the right project names (eg picks 'ant' instead of 'hive-ant').

...

Therefore

...

it's

...

recommended

...

to

...

have

...

the

...

workspace

...

directory

...

one

...

up

...

from

...

the

...

git

...

directory.

...

For

...

example

...

workspaces/hive-workspace/hive

...

where

...

hive-workspace

...

is

...

the

...

eclipse

...

workspace

...

and

...

hive

...

is

...

the

...

git

...

base

...

directory.

...

How

...

to

...

generate

...

tarball?

...

MVN:

{
Code Block
}
mvn clean package -DskipTests -Pdist
{code}

It

...

will

...

then

...

be

...

located

...

in

...

packaging/target/

...

How

...

to

...

generate

...

protobuf

...

code?

...

MVN:

{
Code Block
}
cd ql
mvn clean install -DskipTests -Phadoop-1,protobuf
{code}

h3. How 

How to generate thrift code?

MVN:

Code Block
to generate thrift code?

MVN:
{code}
mvn clean install -Phadoop-1,thriftif -DskipTests -Dthrift.home=/usr/local
{code}

h3. How to compile ODBC?

MVN:
{code}

How to compile ODBC?

MVN:

Code Block
cd odbc
mvn compile -Podbc -Dthrift.home=/usr/local -Dboost.home=/usr/local
{code}


h3. How do 

How do I publish hive artifacts to my local maven repository?

Code Block
I publish hive artifacts to my local maven repository?

{code}
ant package
ant -Dmvn.publish.repo=local maven-build
ant -Dmvn.publish.repo=local maven-publish
{code}


MVN:

{
Code Block
}
mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests
{code}

h2. Testing

Testing

How do I run a single test?

Warning
titleITests

Note that any test in the itests directory needs to be executed from with the itests directory. The pom is disconnected from the parent project for technical reasons.

Single test class:

No Format

h3. How do I run a single test?

{warning:title=ITests}
Note that any test in the itests directory needs to be executed from with the itests directory. The pom is disconnected from the parent project for technical reasons.
{warning}

Single test class:
{noformat}
mvn test -Dtest=ClassName
{noformat}

Single

...

test

...

method:

{
No Format
}
mvn test -Dtest=ClassName#methodName
{noformat}

h3. How do I debug into a single test in Eclipse?

You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described [here|h3. How do I import into eclipse?]. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do {{Run->Debug}}.

h3. A test fails with a NullPointerException in MiniDFSCluster

If any test fails with the error below it means you have an inappoprate umask setting. It should be set to 0022.

{noformat}

How do I debug into a single test in Eclipse?

You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described here. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do Run->Debug.

A test fails with a NullPointerException in MiniDFSCluster

If any test fails with the error below it means you have an inappoprate umask setting. It should be set to 0022.

No Format
java.lang.NullPointerException: null
    at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:426)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:284)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)

How do I run all of the unit tests?

Make sure that your JAVA_HOME is appropriately set (some tests need this), and set ANT_OPTS to increase the size allocated to the Permanent Generation as per the following:

Code Block
{noformat}

h3. How do I run all of the unit tests?

Make sure that your JAVA_HOME is appropriately set (some tests need this), and set ANT_OPTS to increase the size allocated to the Permanent Generation as per the following:

{code}
export ANT_OPTS="-XX:MaxPermSize=512m"
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=256M"
{code}

Then,

...

for

...

a

...

clean

...

build,

...

run

{
Code Block
}
ant clean package test
{code}

MVN:

{
Code Block
}
mvn test
cd itests 
mvn test
{code}

Note

...

that

...

running

...

ant

...

test

...

will

...

not

...

work;

...

ant

...

package

...

does

...

some

...

setup

...

work

...

that

...

is

...

required

...

for

...

the

...

testcases

...

to

...

run

...

successfully.

...

MVN:

...

Note

...

that

...

you

...

need

...

to

...

have

...

previously

...

built

...

and

...

installed

...

the

...

jars:

{
Code Block
}
mvn clean install -DskipTests
cd itests 
mvn clean install -DskipTests

How do update the output of a CliDriver testcase?

Code Block
{code}

h3. How do update the output of a CliDriver testcase?

{code}
ant test -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true
{code}

MVN:

{
Code Block
}
cd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true
{code}

As

...

of

...

Hive

...

0.11.0+

...

you

...

can

...

cut

...

this

...

time

...

in

...

half

...

by

...

specifying

...

that

...

only

...

the

...

ql

...

module

...

needs

...

to

...

rebuild

{
Code Block
}
ant test -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true
{code}

h3. How do I run the 

How do I run the clientpositive/clientnegative

...

unit

...

tests?

...

All

...

of

...

the

...

below

...

require

...

that

...

you

...

have

...

previously

...

run

...

ant

...

package

...

.

...

To

...

run

...

clientpositive

...

tests

{
Code Block
}
ant -Dtestcase=TestCliDriver test
{code}

MVN:

{
Code Block
}
cd itests/qtest
mvn test -Dtest=TestCliDriver
{code}

To

...

run

...

a

...

single

...

clientnegative

...

test

...

alter1.q

{
Code Block
}
ant -Dtestcase=TestNegativeCliDriver -Dqfile=alter1.q test
{code}

MVN:

{
Code Block
}
cd itests/qtest
mvn test -Dtest=TestNegativeCliDriver -Dqfile=alter1.q
{code}

To

...

run

...

all

...

of

...

the

...

clientpositive

...

tests

...

that

...

match

...

a

...

regex,

...

e.g.

...

the

...

partition_wise_fileformat

...

tests

{
Code Block
}
ant -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test
{code}

MVN:

{
Code Block
}
cd itests/qtest
mvn test -Dtest=TestCliDriver -Dqfile_regex=partition_wise_fileformat.*
{code}

To

...

run

...

a

...

single

...

contrib

...

test

...

alter1.q

...

and

...

overwrite

...

the

...

result

...

file

{
Code Block
}
ant -Dtestcase=TestContribCliDriver -Dqfile=alter1.q -Doverwrite=true test
{code}

MVN:

{
Code Block
}
cd itests/qtest
mvn test -Dtest=TestContribCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true
{code}

To

...

run

...

a

...

single

...

test

...

groupby1.q

...

and

...

output

...

detailed

...

information

...

during

...

execution

{
Code Block
}
ant -Dtestcase=TestCliDriver -Dqfile=groupby1.q -Dtest.silent=false test
{code}

As

...

of

...

Hive

...

0.11.0+

...

you

...

can

...

cut

...

down

...

the

...

total

...

build

...

time

...

by

...

specifying

...

that

...

only

...

the

...

ql

...

module

...

needs

...

to

...

rebuild.

...

e.g.

...

run

...

all

...

the

...

partition_wise_fileformat

...

tests

{
Code Block
}
ant -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test
{code}