You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 20 Next »

Background

Hive build jobs include Hive PreCommit Testing and scheduled builds of various branches with various flags.  They are run on the Hive PTest2 Infrastructure, which is an EC2 cluster currently sponsored by Cloudera.  The Hive PTest2 Infra runs each build sequentially, but splits the testing across all the resources of the cluster to achieve much higher throughput.

Location

The infrastructure master is at ec2-174-129-184-35.compute-1.amazonaws.com.  Committers can be granted access to this host by request, see HIVE-4739.

Processes

The infrastructure master hosts two processes:

  1. Jenkins: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/
    • This is the admin for Hive builds.  The dashboard shows the state of the build queue, test results of past builds, etc.  
    • Builds are submitted either manually in the dashboard UI, automatically by patch upload (see Hive PreCommit Testing), or by Jenkins schedule (branch builds).
  2. Hive PTest2 WebServer
    • Exposes a REST API to take test requests from the Jenkins builds.  Each request specifies a profile property-file, which contains a set of properties such as what branch to build and what tests to run.
    • On a test request:
      • Creates EC2 slaves, which are spot-instances.
      • Reads the specified property-file and compiles Hive using it.
      • Distributes the compiled artifacts across the EC2 slaves, makes SSH calls to run the tests remotely in parallel, and gathers the results.
      • If no further test request comes in 30 minutes, the slaves are shutdown.

Jenkins Configuration/Debug

  • Jenkins logs:  /var/log/jenkins/jenkins.log
  • Jenkins config: /etc/sysconfig/jenkins
  • Jenkins home: /var/lib/jenkins  (job info is located at /var/lib/jenkins/jobs)

Hive PTest2 WebServer Configuration/Debug

  • Test Logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/  (redirects to /data/hive-ptest/logs/)
  • Profile properties files: /usr/local/hiveptest/etc/public/
  • Webservice logs: /home/hiveptest/apache-tomcat-7.0.41/logs/
  • Working Directory: /data/hive-ptest/working/ (where libs, source, etc. are stored)
  • Scripts: /usr/local/hiveptest/bin/
  • Relevant JIRAs: HIVE-4675 and HIVE-4739

FAQ:

How do I restart Jenkins?

sudo service jenkins restart

How do I setup a new branch build?

In /usr/local/hiveptest/etc/public you can copy an existing profile property-file of the form {profile.properties}, and update the "repository" property in the new profile to point to the new branch.  Then in Jenkins, copy an existing job, and in the build config modify the profile argument in the Hive PTest2 WebServer call (it will be like: "java ... org.apache.hive.ptest.api.client.PTestClient --profile trunk-mr1") to point to the new profile.

How can I read or make changes to Hive PTest2 Infra code?

Code is located in the Hive git repo.  It is under /hive/testutils/ptest2.  As the Hive PTest2 WebServer is a running process, if changes are made, it needs to be restarted.

How do I stop, start, restart the Hive PTest2 WebServer, or have it use the latest test infra code?

$ sudo /usr/local/hiveptest/bin/stop-server.sh 
$ sudo /usr/local/hiveptest/bin/start-server.sh 
$ sudo /usr/local/hiveptest/bin/restart-server.sh 
$ sudo /usr/local/hiveptest/bin/update.sh

How do I change a build to run a new MiniMr/TezDriver q-test?

See: MiniMR+and+PTest2.

This is all same as configuring any build profile property, located at /usr/local/hiveptest/etc/public.  Find the profile properties of your build (for example the pre-commit tests are using "trunk-mr2.properties), and edit "qFileTests.miniXXXDriver.groups...." to include the name of the new q-test.  TODO- link to more complete instructions.

 

  • No labels