Page History

Infrastructure, Simulator, Automation , new CI Changes for CloudStack:

Overview: With new modifications in CI, improvements in infrastructure automation like provisioning of hypervisor, auto install management server, enhancements in simulator, changes in test framework and test categorization, we Improvements in Infrastructure Automation, Simulator Enhancements, Marvin Framework enhancements,  Test Categorization, we can now automatically provision a Hypervisor Ex: XenServer, KVM etc, auto Install management server, automatically install Marvin, run parallel tests for a given build across multiple hypervisors, automatically log bugs, ,log analysis, generate reports and email with detailed inforamtion etc is possible.  With this, CI will automatically kick start every few hours, picking up latest code, does all the things mentioned, can now run tests faster, in parallel and get CI with automated emailed notifications with minimal effort.

With new self service changes, any body can request an automation run and get the report emailed to them easily. This will help them in validating their feature mergers and other easy validations.(Phase 2)

...

Automation Infrastructure Enhancements:

• Ability to install, run simulator and other hyper visors in different zones simultaneously. Run tests in parallel against multiple hypervisors, EX: Xenserver and KVM regression can now run in parallel,  provision easy automatic provisioning of kvm, xen and other hypervisors now available.

...

• Automatic bug filing system.

• Better utilization of test hardware: Simulator is run in a VM, real hardware is used only for provisioning tests.

...

QA\Team Responsibilities in CI Run:  Once QA\Team identifies failures in the mail report sent above, and logs bugs for the failures, then they can mark individual test case with the logged bugid information. EX: If a test case by name “test_deploy_vm_start_failure” fails under test suite “test_deploy_vm.py” , the  it then  this test case need to be marked with bugid and push information as mentioned below. Push these changes as belowto repo. So, next time, when CI runs, this will disable test case will be disabled automatically from run in CI, till it gets fixed.

@attr(tags = ['selfservice'],BugId="234")

...

Developers Responsibilities in CI Run: Developers, once they have a bug raised for failures in CI, have to fix the bug, can and then remove the bugId information for from the respective test case. Once disabled, it the test case will again be part of CI run.

CI Run: In order to exclude failed test cases from running (for which there are bugs logged already, then while ) from re running again and again,  then while we are running CI, we have to add use below additional tags for bugId to exclude the test cases having bugslogged already from running as to nosetests as mentioned below.

nosetests --with-xunit --xunit-file=test_deploy_vm.xml --with-marvin --marvin-config=/home/santhosh/softwares/cs_4_4/cloudstack/setup/dev/advanced.cfg /home/santhosh/softwares/cs_4_4/cloudstack/test/integration/smoke/test_deploy_vm.py -a tags=selfserviceadvanced,'!BugId' --zone="Sandbox-simulator" --collect-only

...