Page History

...

The qfile tests in itests require the packaging phase. The Maven test phase is after compile and before packaging. We could change the qfile tests to run during the integration-test phase using the "failsafe" plugin but the "failsafe" plugin is different than surefire and IMO is hard to use. If you'd like to give that a try, by all means, go ahead.

How do I debug into a single test in Eclipse?

You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described here. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do Run->Debug.

For more information about debugging, see Debugging Hive Code in the Developer Guide.

A test fails with a NullPointerException in MiniDFSCluster

...

Upload the exact same patch again to the JIRA.

Debugging

How do I debug into a single test in Eclipse?

You can debug into a single JUnit test in Eclipse by first making sure you've built the Eclipse files and imported the project into Eclipse as described here. Then set one or more breakpoints, highlight the method name of the JUnit test method you want to debug into, and do Run->Debug.

Another useful method to debug these tests is to attach a remote debugger. When you run the test, enable the debug mode for surefire by passing in the "-Dmaven.surefire.debug". Additional details on how to turning on debugging for surefire can be found here. Now when you run the tests, it will now wait with a message similar to

Listening for transport dt_socket at address: 5005

Note that this assumes that you are still using the default port 5005 for surefire. Otherwise you might see a different port. Once you see this message, in eclipse right click on the project you want to debug, go to "Debug As -> Debug Configurations -> Remote Java Application" and hit the "+" sign on far left top. This should bring up a dialog box. Make sure that the host is "localhost" and the port is "5005". Before you start debugging, make sure that you have set appropriate debug breakpoints in the code. Once ready, hit "Debug". Now if you go back to the terminal, you should see the tests running and will stop at the breakpoints that you set for you to debug.

How do I debug my queries in hive?

You can also interactively debug your queries in hive. by attaching a remote debugger. To do so, start beeline with "–debug" option.

$ beeline --debug
Listening for transport dt_socket at address: 8000

Once you see this message, in eclipse right click on the project you want to debug, go to "Debug As -> Debug Configurations -> Remote Java Application" and hit the "+" sign on far left top. This should bring up a dialog box. Make sure that the host is the host on which beeline CLI is running and the port is "8000". Before you start debugging, make sure that you have set appropriate debug breakpoints in the code. Once ready, hit "Debug". The remote debugger should attach to beeline and proceed.

$ beeline --debug
Listening for transport dt_socket at address: 8000
Beeline version 1.2.0 by Apache Hive
beeline>

At this point, run your queries as normal and it should stop at the breakpoints that you set so that you can start debugging.

This method should work great if your queries are simple fetch queries do not kick off mapreduce jobs. If a query runs in a distributed mode, it becomes very hard to debug. Therefore, it is advisable to run in a "local" mode for debugging. In order to run hive in local mode, do the following:

MRv1:

SET mapred.job.tracker=local

MRv2 (YARN):

SET mapreduce.framework.name=local

At this point, attach the remote debugger as mentioned before to start debugging your queries.