You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

"My company has a xyz feature for CloudStack. Can someone help me test it?"

This page is a proposal for creating a unified design for QA testing and automation. It should be looked at as a blueprint to follow for a QA set-up (think QACloud like DevCloud)

Any comments / suggestions will be much appreciated.

Goals

  1. Enable community to run QA tests in automated way
  2. Make it easy to replicate a set-up
  3. Enable to run BVTs continuously

It is important to note that keeping it simple and easy to replicate is one of the main goals of this exercise. Thus, it may not cater to every possible use case.
The code is based on Prasanna's effort on similar lines (big thanks to Prasanna!)

Requirements

  1. Every test run should be isolated from previous test run
  2. Clean-up hypervisors between runs
  3. Spawn up new management server for each run

Infrastructure

Every single set-up would contain

  1. XenServer to host VM for Cobbler etc., and CloudStack management server
  2. Isolated private network that is RFC1918 compliant. ( Recommend to use 172.16.0.0/16 subnet so it is easier to follow configuration). This allows CloudStack to run in isolation
  3. Three baremetal hosts to serve as the "Cloud"
  4. Dedicated IPMI network for baremetal provisioning

At the heart is a VM that serves as a Jenkins slave, and contains a set of tools to allow one to run QA tests on a clean environment. These tools, hosted on the VM (lets call it "Provisioner"), are :

Tool / Software

Use

Comments

Cobbler

Provision baremetal hosts with hypervisor
Provision management server OS

Cobbler integrates well with Puppet and provides useful helper scripts

Puppet

Configure management server with mysql etc

 

DNSMASQ

DHCP + DNS

Used to provision hosts

Squid

HTTP Proxy

Testing infrastructure uses isolated network. Some tasks, like yum install, need internet access

NFS Server

As NFS secondary storage

Packeged systemVM and builtin templates for quicker download

ipmitool

Interact with baremetal machines through IPMI network

 

Marvin

Deployment and testing framework for CloudStack

 

The whole code is Python based making it easier to package as needed in future.

A typical deployment would have the Provisioner VM also act as a Jenkins slave for builds and launching test suite

Understanding Process Flow

There are two steps in the QA process :

  1. Set up Cloud
  2. Launch and report test results

Typically, the setting up of the Cloud is a separate Jenkins job, upon successful run of which we launch a test suite in a different job. Note that, the code contains basic elements for you to set-up a single Jenkins job for

both the steps, though it may be very slow (Since BVTs are run sequentially in this case to avoid Marvin issues)

Setting up from scratch

Follow the instructions here to set-up the infrastructure from scratch.

Easy Set-Up

Setting up the infrastructure

  1. Budget at least 4 machines to your set-up. Install XenServer (6+) on one machine to host Provisioner and management server. This machine is required to have internet access.
  2. Set-up and configure IPMI network and enable hosts for IPMI. This link may be useful http://stuff.mit.edu/afs/athena/dept/cron/documentation/dell-server-admin/en/DRAC_5/racugc1j.htm
  3. Set-up private switch for hosts on eth0. Reference config is available here
  4. Import Provisioner template by downloading from here
  5. Change MAC address in Cobbler (cobbler system edit --mac-address=<MAC> --name=xen1/2/3 ), DNSMASQ (/etc/dnsmasq.conf) and system.properties file
  6. Set IPMI password in configure script
  7. Set-up public internet access for Provisioner VM so it may proxy requests as needed.
Advanced Options
  1.  If the switch config does not  mimic provided reference, please change the IP address for DNSMASQ and system.properties file
  2. Set correct routes for IP configuration in Provisioner and its host XenServer

The configuration would look something like the below after things are correctly set-up, after which you should proceed to adding Jenkins jobs as described in next section.

Jenkins Jobs

Set-up Jenkins job for infrastructure provisioning. This is a simple jenkins job that packages CloudStack RPMs and launches a tester.py python script. The following configuration may be used :

echo export M2_HOME=/usr/local/apache-maven-3.1.0
echo export M2=/usr/local/apache-maven-3.1.0/bin
echo export PATH=/usr/local/apache-maven-3.1.0/bin:$PATH
PACKAGE_NAME="CloudStack_Auto-rhel6.3"
cd $WORKSPACE
rm -rf dist/rpmbuild/*
cd packaging/centos63
./package.sh
cd ../..
tempdir=`mktemp -d`
mkdir -p "$tempdir"
cp dist/rpmbuild/RPMS/x86_64/*.rpm $tempdir/
createrepo $tempdir/mv $tempdir $PACKAGE_NAME
tar -cvzf $PACKAGE_NAME.tar.gz $PACKAGE_NAME
cd /root/cloud-autodeploypython2.7 /root/cloud-autodeploy/tester.py $WORKSPACE xen
  • No labels