You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

As discussed in HIVE-2985 (http://issues.apache.org/jira/browse/HIVE-2985), the goal of this exercise is to create a end to end test framework.

Currently, Facebook is using a deployment of hive which is pretty close to trunk. We would like to continue doing so, but would like to invest more
in a test framework in order to do so.

The high level idea is to replicate the deployment framework from Facebook.

This will us get the changes tested thoroughly in our environment before they are committed. Also, it make easier for contributors outside Facebook to test/debug their changes in this environment and make sure they are not breaking anything.

It would be perfect to get all changes to the apache hive trunk run a subset of Facebook workload on a test cluster (inside Facebook) using the deployment model
in Facebook (including all the custom hooks, and the configurations) before being committed. This would give us the confidence that none of these changes are breaking
any internal Facebook deployment. However, if any test breaks, and the patch is being contributed by an outside Facebook contributor, it is impossible for the
outside committer to debug these issues, since there is no access to this test environment. The work-around is to mimic this workload, and make it available in open
source, so that all committers can run/debug these tests.

This document lists the steps in detail of how to get there:

  • Create a set of tests which are run in parallel, on a real hadoop cluster, using apache hive trunk and the version of hadoop used by facebook

Run the apache hive trunk

  • Make the hooks available used by Facebook be used in these tests.
  • No labels