...
Hive
...
Developer
...
FAQ
...
Table of Contents |
---|
Info | ||
---|---|---|
| ||
Hive is using Maven as its build tool. Versions prior to 0.13 were using Ant. |
Developing
How do I move some files?
Post a patch for testing purposes which simply does add and deletes. SVN will not understand these patches are actually moves, therefore you should actually upload the following, in order so the last upload is the patch for testing purposes:
- A patch which has only the non-move changes for commit e.g. HIVE-XXX-for-commit.patch
...
- A
...
- script
...
- of
...
- of
...
- commands
...
- required
...
- to
...
- make
...
- the
...
- moves
...
- HIVE-XXX-moves.sh
...
- A
...
- patch
...
- for
...
- testing
...
- purposes
...
- HIVE-XXX.patch
...
The
...
script
...
should
...
be
...
a
...
set
...
of
...
svn
...
mv
...
commands
...
along
...
with
...
any
...
perl
...
commands
...
required
...
for
...
find/replace.
...
E.g.:
No Format |
---|
{format} $ svn mv MyCLass.java MyClass.java $ perl -i -pe 's@MyCLass@MyClass@gs<at:var at:name="MyCLass" />MyClass@g' MyClass.java {format} h2. Building h3. How do I rerun precommit tests over the same patch? Upload the exact same patch again to the JIRA. h3. Maven settings You might have to set the following maven options on certain systems to get build working. Check out the suggested value for MAVEN_OPTS |
Building
How do I rerun precommit tests over the same patch?
Upload the exact same patch again to the JIRA.
Maven settings
You might have to set the following maven options on certain systems to get build working. Check out the suggested value for MAVEN_OPTS under the testing section.
How to build all source?
MVN:
Code Block |
---|
under the testing section. h3. How to build all source? MVN: {code} mvn clean install -DskipTests cd itests mvn clean install -DskipTests {code} h3. How do I import into eclipse? Build and generate eclipse files (the conservative method): { |
How do I import into eclipse?
Build and generate eclipse files (the conservative method):
Code Block |
---|
code} $ mkdir workspace $ cd workspace $ git clone https://github.com/apache/hive.git $ cd hive $ mvn clean install -DskipTests $ mvn eclipse:clean $ mvn eclipse:eclipse -DdownloadSources -DdownloadJavadocs $ cd itests $ mvn clean install -DskipTests $ mvn eclipse:clean $ mvn eclipse:eclipse -DdownloadSources -DdownloadJavadocs {code} |
In
...
eclipse
...
define
...
M2_REPO
...
in
...
Preferences
...
->
...
Java
...
->
...
Build
...
Path
...
->
...
Classpath
...
Variables
...
to
...
either:
...
Mac
...
Example:
Code Block |
---|
} /Users/$USER/.m2/repository {code} |
Linux
...
Example:
Code Block |
---|
} /home/$USER/.m2/repository {code} |
Windows
...
Example:
Code Block |
---|
} C:/users/$USER/.m2/repository {code} |
Then
...
import
...
the
...
workspaces.
...
If
...
you
...
an
...
error
...
about
...
"restricted
...
use
...
of
...
Signal"
...
for
...
beeline
...
and
...
cli
...
follow
...
...
...
instructions.
Note that if you use the hive git base dir as eclipse workspace, then it does not pick the right project names (eg picks 'ant' instead of 'hive-ant').
...
Therefore
...
it's
...
recommended
...
to
...
have
...
the
...
workspace
...
directory
...
one
...
up
...
from
...
the
...
git
...
directory.
...
For
...
example
...
workspaces/hive-workspace/hive
...
where
...
hive-workspace
...
is
...
the
...
eclipse
...
workspace
...
and
...
hive
...
is
...
the
...
git
...
base
...
directory.
...
How
...
to
...
generate
...
tarball?
...
MVN:
Code Block |
---|
} mvn clean package -DskipTests -Pdist {code} |
It
...
will
...
then
...
be
...
located
...
in
...
packaging/target/
...
How
...
to
...
generate
...
protobuf
...
code?
...
MVN:
Code Block |
---|
} cd ql mvn clean install -DskipTests -Phadoop-1,protobuf {code} h3. How |
How to generate thrift code?
MVN:
Code Block |
---|
to generate thrift code? MVN: {code} mvn clean install -Phadoop-1,thriftif -DskipTests -Dthrift.home=/usr/local {code} h3. How to compile ODBC? MVN: {code} |
How to compile ODBC?
MVN:
Code Block |
---|
cd odbc
mvn compile -Podbc -Dthrift.home=/usr/local -Dboost.home=/usr/local
{code}
h3. How do |
How do I publish hive artifacts to my local maven repository?
Code Block |
---|
I publish hive artifacts to my local maven repository? {code} ant package ant -Dmvn.publish.repo=local maven-build ant -Dmvn.publish.repo=local maven-publish {code} |
MVN:
Code Block |
---|
} mvn clean install -DskipTests cd itests mvn clean install -DskipTests {code} h2. Testing |
Testing
How do I run a single test?
Warning | ||
---|---|---|
| ||
Note that any test in the itests directory needs to be executed from with the itests directory. The pom is disconnected from the parent project for technical reasons. |
Single test class:
No Format |
---|
h3. How do I run a single test? {warning:title=ITests} Note that any test in the itests directory needs to be executed from with the itests directory. The pom is disconnected from the parent project for technical reasons. {warning} Single test class: {noformat} mvn test -Dtest=ClassName {noformat} |
Single
...
test
...
method:
No Format |
---|
} mvn test -Dtest=ClassName#methodName {noformat} h3. How do I debug into a single test in Eclipse? You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described [here|h3. How do I import into eclipse?]. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do {{Run->Debug}}. h3. A test fails with a NullPointerException in MiniDFSCluster If any test fails with the error below it means you have an inappoprate umask setting. It should be set to 0022. {noformat} |
How do I debug into a single test in Eclipse?
You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described here. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do Run->Debug
.
A test fails with a NullPointerException in MiniDFSCluster
If any test fails with the error below it means you have an inappoprate umask setting. It should be set to 0022.
No Format |
---|
java.lang.NullPointerException: null at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:426) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:284) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124) |
How do I run all of the unit tests?
Make sure that your JAVA_HOME is appropriately set (some tests need this), and set ANT_OPTS to increase the size allocated to the Permanent Generation as per the following:
Code Block |
---|
{noformat} h3. How do I run all of the unit tests? Make sure that your JAVA_HOME is appropriately set (some tests need this), and set ANT_OPTS to increase the size allocated to the Permanent Generation as per the following: {code} export ANT_OPTS="-XX:MaxPermSize=512m" export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=256M" {code} |
Then,
...
for
...
a
...
clean
...
build,
...
run
Code Block |
---|
} ant clean package test {code} |
MVN:
Code Block |
---|
} mvn test cd itests mvn test {code} |
Note
...
that
...
running
...
ant
...
test
...
will
...
not
...
work;
...
ant
...
package
...
does
...
some
...
setup
...
work
...
that
...
is
...
required
...
for
...
the
...
testcases
...
to
...
run
...
successfully.
...
MVN:
...
Note
...
that
...
you
...
need
...
to
...
have
...
previously
...
built
...
and
...
installed
...
the
...
jars:
Code Block |
---|
}
mvn clean install -DskipTests
cd itests
mvn clean install -DskipTests
|
How do update the output of a CliDriver testcase?
Code Block |
---|
{code} h3. How do update the output of a CliDriver testcase? {code} ant test -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true {code} |
MVN:
Code Block |
---|
} cd itests/qtest mvn test -Dtest=TestCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true {code} |
As
...
of
...
Hive
...
0.11.0+
...
you
...
can
...
cut
...
this
...
time
...
in
...
half
...
by
...
specifying
...
that
...
only
...
the
...
ql
...
module
...
needs
...
to
...
rebuild
Code Block |
---|
} ant test -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile=alter1.q -Doverwrite=true {code} h3. How do I run the |
How do I run the clientpositive/clientnegative
...
unit
...
tests?
...
All
...
of
...
the
...
below
...
require
...
that
...
you
...
have
...
previously
...
run
...
ant
...
package
...
.
...
To
...
run
...
clientpositive
...
tests
Code Block |
---|
} ant -Dtestcase=TestCliDriver test {code} |
MVN:
Code Block |
---|
} cd itests/qtest mvn test -Dtest=TestCliDriver {code} |
To
...
run
...
a
...
single
...
clientnegative
...
test
...
alter1.q
Code Block |
---|
} ant -Dtestcase=TestNegativeCliDriver -Dqfile=alter1.q test {code} |
MVN:
Code Block |
---|
} cd itests/qtest mvn test -Dtest=TestNegativeCliDriver -Dqfile=alter1.q {code} |
To
...
run
...
all
...
of
...
the
...
clientpositive
...
tests
...
that
...
match
...
a
...
regex,
...
e.g.
...
the
...
partition_wise_fileformat
...
tests
Code Block |
---|
} ant -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test {code} |
MVN:
Code Block |
---|
} cd itests/qtest mvn test -Dtest=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* {code} |
To
...
run
...
a
...
single
...
contrib
...
test
...
alter1.q
...
and
...
overwrite
...
the
...
result
...
file
Code Block |
---|
} ant -Dtestcase=TestContribCliDriver -Dqfile=alter1.q -Doverwrite=true test {code} |
MVN:
Code Block |
---|
} cd itests/qtest mvn test -Dtest=TestContribCliDriver -Dqfile=alter1.q -Dtest.output.overwrite=true {code} |
To
...
run
...
a
...
single
...
test
...
groupby1.q
...
and
...
output
...
detailed
...
information
...
during
...
execution
Code Block |
---|
} ant -Dtestcase=TestCliDriver -Dqfile=groupby1.q -Dtest.silent=false test {code} |
As
...
of
...
Hive
...
0.11.0+
...
you
...
can
...
cut
...
down
...
the
...
total
...
build
...
time
...
by
...
specifying
...
that
...
only
...
the
...
ql
...
module
...
needs
...
to
...
rebuild.
...
e.g.
...
run
...
all
...
the
...
partition_wise_fileformat
...
tests
Code Block |
---|
} ant -Dmodule=ql -Dtestcase=TestCliDriver -Dqfile_regex=partition_wise_fileformat.* test {code} |