Page History

...

marvin-nose - marvin plugin for nose

should_dsl - should assertions dsl-style

Windows specifics

For Windows (non-cygwin) development environment, following additional steps are needed to be able to install Marvin:

...

The developer profile compiles and packages Marvin. It does NOT install marvin your default PYTHONPATH.

Code Block

language	bash
title	building marvin

bash
$ mvn -P developer -pl :cloud-marvin

Alternately, you can fetch marvin from jenkins as well: packages are built every hour here

Installation

Code Block

language	bash
title	fresh install of marvin


$ pip install tools/marvin/dist/Marvin-0.1.0*.tar.gz

To upgrade an existing marvin installation and sync it with the latest APIs on the managemnet server, follow the steps herecall this command

Code Block

language	bash
title	upgrade of an existing marvin installation from source

 $ pip install --upgrade tools/marvin/dist/Marvin-*.tar.gz

First Things First

You will need a CloudStack management server that is already deployed, configured and ready to accept API calls. You can pick any management server in your lab that has a few VMs running on it or you can use DevCloud or the simulator environment deployed for a checkin test #checkin. Create a sample json config file demo.cfg telling marvin where your management server and database server are.

The demo.cfg json config looks as shown below:

Code Block

title	an example environment definition

json
{
    "dbSvr": {
        "dbSvr": "marvin.testlab.com",
        "passwd": "cloud",
        "db": "cloud",
        "port": 3306,
        "user": "cloud"
    },
    "logger": {
         "LogFolderPath": "/tmp/"
    },
    "mgtSvr": [
        {
            "mgtSvrIp": "marvin.testlab.com",
            "port": 8096,
            "user": "root",
            "passwd": "password",
            "hypervisor": "XenServer",
        }
    ]
}

...

- open up the integration.port on your management server iptables -I INPUT -p tcp dport 8096 -j ACCEPTas root user.

Code Block

language	bash
title	open access to your management server

$ sudo iptables -I INPUT -p tcp --dport 8096 -j ACCEPT

- - Change the global setting integration.api.port on the CloudStack GUI to 8096 and restart the management server

...

tearDown the user account - basically delete it to cleanup acquired resources
If you have test data pertaining to a test suite, use the file under tools/marvin/marvin/config/test_data.py. Please refer existing smoke test suites for the same.

Here is our test_deploy_vm.py modulemodule:

Code Block

python
#All tests inherit from cloudstackTestCase
from marvin.cloudstackTestCase import cloudstackTestCase

#Import Integration Libraries

#base - contains all resources as entities and defines create, delete, list operations on them
from marvin.integration.lib.base import Account, VirtualMachine, ServiceOffering

#utils - utility classes for common cleanup, external library wrappers etc
from marvin.integration.lib.utils import cleanup_resources

#common - commonly used methods for all tests are listed here
from marvin.integration.lib.common import get_zone, get_domain, get_template


class TestDataTestDeployVM(objectcloudstackTestCase):
    """Test datadeploy objecta thatVM isinto requireda touser create resourcesaccount
    """

 
	@classmethod
    def __init__(selfsetUpClass(cls):
        self.testdata = {super(TestDeployVM, cls)

 
 
    def setUp(self):
       #data toself.apiclient create an account= self.testClient.getApiClient()
        self.testdata = self.testClient.getParsedTestDataConfig()
 

 "account": {
      # Get Zone, Domain and Default Built-in template
   "email": "test@test.com",
    self.domain = get_domain(self.apiclient)
        self.zone  "firstname": "Test",
 = get_zone(self.apiclient, self.testClient.getZoneForTests())
 

               "lastname": "User",self.testdata["mode"] = self.zone.networktype
        self.template = get_template(self.apiclient, self.zone.id, self.testdata["ostype"])
 

   "username": "test",
    #create a user account
         "password": "password",self.account = Account.create(
            }self.apiclient,
            #data reqd for virtual machine creation
self.testdata["account"],
            domainid=self.domain.id
      "virtual_machine" : {)

        #create a service offering
        small_service_offering = "name" : "testvm",self.testdata["service_offerings"]["small"]
        small_service_offering['storagetype'] = 'local'
       "displayname" : "Test VM", self.service_offering = ServiceOffering.create(
            }self.apiclient,
            #small small_service _offering
        )
    "service_offering": {
   #build cleanup list
        self.cleanup = [
 "small": {
          self.service_offering,
          "name": "Small Instance", self.account
        ]
 
 
	def tearDown(self):
        try:
   "displaytext": "Small Instance",
       cleanup_resources(self.apiclient, self.cleanup)
        except Exception as  "cpunumber": 1,e:
            self.debug("Warning! Exception in tearDown: %s" % e)
 
 
     "cpuspeed": 100,
def test_deploy_vm(self):
        """Test Deploy Virtual Machine

        # Validate the "memory": 256,following:
        # - listVirtualMachines returns accurate information
    },
    """
        },
  self.virtual_machine = VirtualMachine.create(
          "ostype": 'CentOS 5.3 (64-bit)'self.apiclient,
        }


class TestDeployVM(cloudstackTestCase):
    self.testdata["""Test deploy a VM into a user account
    """
virtual_machine"],
            accountid=self.account.name,
    def setUp(self):
        zoneid=self.testdata = TestData().testdata
.zone.id,
           self.apiclient domainid= self.testClient.getApiClient()

account.domainid,
        # Get Zone, Domain and Default Built-in template
 serviceofferingid=self.service_offering.id,
           self.domain templateid= get_domain(self.apiclient, self.testdata)template.id
        )

  self.zone = get_zone     list_vms = VirtualMachine.list(self.apiclient, id=self.testdatavirtual_machine.id)

        self.testdata["mode"] = self.zone.networktypedebug(
        self.template = get_template(self.apiclient, self.zone.id, self.testdata["ostype"])

        #create a user account
"Verify listVirtualMachines response for virtual machine: %s"\
            % self.account = Account.create(
.virtual_machine.id
        )

        self.apiclient,assertEqual(
            self.testdata["account"]isinstance(list_vms, list),
            domainid=self.domain.idTrue,
        )
    "List VM response was #createnot a servicevalid offeringlist"
        self.service_offering = ServiceOffering.create(
    )
        self.apiclient,assertNotEqual(
            self.testdata["service_offering"]["small"]len(list_vms),
        )
    0,
    #build cleanup list
      "List VM self.cleanupresponse =was [empty"
            self.service_offering,)

        vm    self.account= list_vms[0]
        ]
self.assertEqual(
    def test_deploy_vm(self):
        """Test Deploy Virtual Machine

vm.id,
            # Validate the following:
self.virtual_machine.id,
          # 1. "Virtual Machine isids accessibledo vianot SSHmatch"
        #)
 2. listVirtualMachines returns accurate information
   self.assertEqual(
     """
        self.virtual_machine = VirtualMachine.create(vm.name,
            self.virtual_machine.apiclientname,
            self.testdata["virtual_machine"],
    "Virtual Machine names do not match"
        accountid=self.account.name,
   )
         zoneid=self.zone.id,assertEqual(
            domainid=self.account.domainidvm.state,
            serviceofferingid=self.service_offering.id"Running",
            templateid=self.template.id
        )

   msg="VM is not in Running state"
     list_vms = VirtualMachine.list(self.apiclient, id=self.virtual_machine.id)

        self.debug(
            "Verify listVirtualMachines response for virtual machine: %s"\
            % self.virtual_machine.id
        )

        self.assertEqual(
            isinstance(list_vms, list),
            True,
            "List VM response was not a valid list"
        )
        self.assertNotEqual(
            len(list_vms),
            0,
            "List VM response was empty"
        )

        vm = list_vms[0]
        self.assertEqual(
            vm.id,
            self.virtual_machine.id,
            "Virtual Machine ids do not match"
        )
        self.assertEqual(
            vm.name,
            self.virtual_machine.name,
            "Virtual Machine names do not match"
        )
        self.assertEqual(
            vm.state,
            "Running",
            msg="VM is not in Running state"
        )

    def tearDown(self):
        try:
            cleanup_resources(self.apiclient, self.cleanup)
        except Exception as e:
            self.debug("Warning! Exception in tearDown: %s" % e)

Parts of the test

imports

cloudstackTestCase - All the cloudstack marvin tests inherit from this class. The class provides the tests with apiclient and dbclients using which calls can be made to the API server. dbclient is useful for verifying database values using SQL

Parts of the test

imports

cloudstackTestCase - All the cloudstack marvin tests inherit from this class. The class provides the tests with apiclient and dbclients using which calls can be made to the API server. dbclient is useful for verifying database values using SQL

lib.base - the base module contains all the resources one can manipulate in cloudstack. eg: VirtualMachine, Account, Zone, VPC, etc. Each resource defines a set of operations. For eg: VirtualMachine.deploy, VirtualMachine.destroy, VirtualMachine.list, etc. Each operation makes an API call: For eg: VirtualMachine.recover -> recoverVirtualMachineCmd integration.lib.base - the base module contains all the resources one can manipulate in cloudstack. eg: VirtualMachine, Account, Zone, VPC, etc. Each resource defines a set of operations. For eg: VirtualMachine.deploy, VirtualMachine.destroy, VirtualMachine.list, etc. Each operation makes an API call: For eg: VirtualMachine.recover -> recoverVirtualMachineCmd returning a recoverVirtualMachineResponse. When dealing with the tests one only has to identify the set of resources one will be using and all the operations will autocomplete if you are using an IDE like PyDev/PyCharm

integration.lib.utils - utility classes for performing verification/actions outside the API. Eg. ssh utilities, random_string generator, cleanup etc.

integration.lib.common - simple methods that are commonly required for most of the tests, eg: a template -get_template, a zone-get_zone, etc

...

The test data class carries information in a dictionary object. (key, value) pairs in this class are needed to be externally supplied to satisfy an API call. For eg: In order to create a VM one needs to give a displayname and the vm name. These are externally supplied data. It is not mandatory to use the testdata class to supply to your test. In all cases you can simply send the right arguments to the Resource.operation( method of your resource without using testdata dictionaries. The advantage of testdata is keeping all data to be configurable in a single place.

Although test data is at the top of our test class, it is only identified as and when we start writing our testMention all test data related to a test suite under tools/marvin/marvin/config/test_data.py. In our case we have identified that we need an account (firstname,lastname etc), a virtual_machine (with name and displayname) and a service_offering (with cpu: 128 and some memory) as test data. Please refer to current test suites under smoke for specific examples

TestDeployVM

TestDeployVM is our test class that holds the suite of tests we want to perform. All test classes start with Capital caseing and the Test prefix. Ideally only one test class is contained in every module

...

self.cleanup =[]. The cleanup list contains the list of objects which should be destroyed at the end of our test run in the tearDown method

tearDown() - the teardown method

tearDown() - the teardown method simply calls the cleanup (delete) associated with every resource thereby garbage collecting resources of the test

test_deploy_vm - our test scenario. All methods must begin with the test_ prefix

VirtualMachine.creat( -> this is the deployVirtualMachineCmd)

triple quote comments talk about the scenario in detail

Multiple asserts are made with appropriate failure messages

Read up about asserts here

...

simply calls the cleanup (delete) associated with every resource thereby garbage collecting resources of the test

test_deploy_vm - our test scenario. All methods must begin with the test_ prefix

VirtualMachine.creat( -> this is the deployVirtualMachineCmd)

triple quote comments talk about the scenario in detail

Multiple asserts are made with appropriate failure messages

Read up about asserts here

You can also use the should_dsl to do asserts as it is included with the Marvin install and is more readable

Test Categories

Marvin supports test categories which enables you to run specific tests for product areas. For example if you have made a change in the accounts product area, there's a way to trigger all accounts related tests in both smoke and component tests directories. Just put a tag 'accounts' and point the directories/files

Example:

Info
nosetests --with-marvin --marvin-config=[config] --hypervisor=xenserver -a tags=accounts [file(s)]

More info on this wiki page: Categories

Running the test

IDE - Eclipse and PyDev

...

--with-marvin
--marvin-config=/path/to/demo.cfg--load

Save/Apply

Now create a Debug Configuration with the project set the one in which you are writing your tests. And the main module to be your test_deploy_vm.py script we defined earlier. Hit Debug and you should see your test run within the Eclipse environment and report failures in the Debug Window. You will also be able to set breakpoints, inspect values, evaluate expressions while debugging like you do with Java code in Eclipse.

...

Code Block
bash tsp@cloud:~/cloudstack# nosetests --with-marvin --marvin-config=demo.cfg --load test_deploy_vm.py Test Deploy Virtual Machine ok ---- Ran 1 test in 10.396s OK

...

Note that the testclient is available from the superclass using getClsTestClient in this case.

Anchor

	checkin
	checkin

Checkin/smoke Tests

The agent simulator and marvin are integrated into maven build phases to help you run basic tests before pushing a commit. These tests are integration tests that will test the CloudStack system as a whole. Management Server will be running during the tests with the Simulator Agent responding to hypervisor commands. For running the checkin tests, your developer environment needs to have Marvin installed and working with the latest CloudStack APIs. These tests are lightweight and should ensure that your commit doesnt break critical functionality for others working with the master branch. The checkin-tests utilize marvin and a one-time installation of marvin will be done so as to fetch all the related dependencies. Further updates to marvin can be done by using the sync mechanism described later in this section. In order for these tests to run on simulator, we need to add an attribute <required_hardware="false"> to these test cases.

These build steps are similar to the regular build, deploydb and run of the management server. Only some extra switches are required to run the tests and should be easy to recall and run anytime:

...

Code Block
$ sudo mvn -Pdeveloper,marvin.sync -Dendpoint=localhost -pl :cloud-marvin

Run the Tests

Once the simulator is up, In a separate session you can use the following two commands to bring up n zone( below example creates an advanced zone with two simulator hypervisors followed by run tests that are tagged to work on the simulator:marvin.setup to bring up a zone with the config specified) and run tests.

command 1: Below command deploys a datacenter.

Code Block
$ mvn -Pdeveloper,marvin.setup -Dmarvin.config=python tools/marvin/marvin/deployDataCenter.py -i setup/dev/advanced.cfg -pl :cloud-marvin integration-test.cfg

Example configs are available in setup/dev/advanced.cfg and setup/dev/basic.cfg

marvin.test to run the checkin tests that are tagged to run on the simulator

...

command 2: Below command runs the test.

Code Block

$ export MARVIN_CONFIG=setup/dev/advanced.cfg
$ export TEST_SUITE=test/integration/smoke
$ export ZONE_NAME=Sandbox-simulator
$ nosetests-2.7 \
  --with-marvin \
  --marvin-config=${MARVIN_CONFIG} \
  -w ${TEST_SUITE} \
  --with-xunit \
  --xunit-file=/tmp/bvt_selfservice_cases.xml \
  --zone=${ZONE_NAME} \
  --hypervisor=simulator \
  -a tags=advanced,required_hardware=false

The --zone argument should match the name of the zone defined in the config file (currently Sandbox-simulator for basic.cfg and advanced.cfg).

Including your own

Check-In tests are the same as any other tests written using Marvin. The only additional step you need to do is ensure that your test is driven entirely by the API only. This makes it possible to run the test on a simulator. Once you have your test, you need to tag it to run on the simulator so the marvin test runner can pick it up during the checkin-test run. Then place your test module in the test/integration/smoke folder and it will become part of the checkin test run.

For eg:

Code Block

Code Block
@attr(tags =["simulatoradvanced", "advancedsmoke"], "smokerequired_hardware="false") def test_deploy_virtualmachine(self): """Tests deployment of VirtualMachine """

Code Block

The sample simulator configurations for advanced and basic zone is available in setup/dev/ directory. The default configuration setup/dev/advanced.cfg deploys an advanced zone with two simulator hypervisors in a single cluster in a single pod, two primary NFS storage pools and a secondary storage NFS store. If your test requires any extra hypervisors, storage pools, additional IP allocations, VLANs etc - you should adjust the configuration accordingly. Ensure that you have run all the checkin tests in the new configuration. For this you can directly edit the JSON file or generate a new configuration file. The setup/dev/advanced.cfg was generated as follows

...

Tests with more backend verification and complete integration of suites for network, snapshots, templates etc can be found in the test/integration/smoke and test/integration/component. Almost all of these test suites use common library wrappers written around the test framework to simplify writing tests. These libraries are part of the marvin.integrationlib package. Ensure that you have gone through the existing tests related to your feature before writing your own.

...

Code Block

tsp@cloud:~/cloudstack# nosetests --with-marvin --marvin-config=tools/devcloud/devcloud.cfg load -a tags='devcloud' test/integration/smoke


Test Deploy Virtual Machine  ok
Test Stop Virtual Machine  ok
Test Start Virtual Machine  ok
Test Reboot Virtual Machine  ok
Test destroy Virtual Machine  ok
Test recover Virtual Machine  ok
Test destroy(expunge) Virtual Machine  ok


----

Ran 7 tests in 10.001s


OK

...

Code Block

json
{
    "zones": [
        {
            "name": "Sandbox-XenServer",
            "guestcidraddress": "10.1.1.0/24",
            "physical_networks": [
                {
                    "broadcastdomainrange": "Zone",
                    "name": "test-network",
                    "traffictypes": [
                        {
                            "typ": "Guest"
                        },
                        {
                            "typ": "Management"
                        },
                        {
                            "typ": "Public"
                        }
                    ],
                    "providers": [
                        {
                            "broadcastdomainrange": "ZONE",
                            "name": "VirtualRouter"
                        }
                    ]
                }
            ],
            "dns1": "10.147.28.6",
            "ipranges": [
                {
                    "startip": "10.147.31.150",
                    "endip": "10.147.31.159",
                    "netmask": "255.255.255.0",
                    "vlan": "31",
                    "gateway": "10.147.31.1"
                }
            ],
            "networktype": "Advanced",
            "pods": [
                {
                    "endip": "10.147.29.159",
                    "name": "POD0",
                    "startip": "10.147.29.150",
                    "netmask": "255.255.255.0",
                    "clusters": [
                        {
                            "clustername": "C0",
                            "hypervisor": "XenServer",
                            "hosts": [
                                {
                                    "username": "root",
                                    "url": "http://10.147.29.58",
                                    "password": "password"
                                }
                            ],
                            "clustertype": "CloudManaged",
                            "primaryStorages": [
                                {
                                    "url": "nfs://10.147.28.6:/export/home/sandbox/primary",
                                    "name": "PS0"
                                }
                            ]
                        }
                    ],
                    "gateway": "10.147.29.1"
                }
            ],
            "internaldns1": "10.147.28.6",
            "secondaryStorages": [
                {
                    "url": "nfs://10.147.28.6:/export/home/sandbox/secondary"
                }
            ]
        }
    ],
    "dbSvr": {
        "dbSvr": "10.147.29.111",
        "passwd": "cloud",
        "db": "cloud",
        "port": 3306,
        "user": "cloud"
    },
    "logger":{
         "LogFolderPath": "/tmp/"
    },
    "globalConfig": [
        {
            "name": "TestClientstorage.cleanup.interval",
            "filevalue": "/var/log/testclient.log300"
        },
        {
            "name": "TestCaseaccount.cleanup.interval",
            "filevalue": "/var/log/testcase.log600"
        }
    ],
    "globalConfigmgtSvr": [
        {
            "namemgtSvrIp": "storage10.147.cleanup29.interval111",
            "valueport": "300"8096
        },
        {
            "name": "account.cleanup.interval",
            "value": "600"
        }
    ],
    "mgtSvr": [
        {
            "mgtSvrIp": "10.147.29.111",
            "port": 8096
        }
    ]
}

What you saw in the beginning of the tutorial was a condensed form of this complete configuration file. If you are familiar with the CloudStack installation you will recognize that most of these are settings you give in the install wizards as part of configuration. What is different from the simplified configuration file are the sections zones and globalConfig. The globalConfig section is nothing but a simple listing of (key, value) pairs for the Global Settings section of CloudStack.

The zones section defines the hierarchy of our cloud. At the top-level are the availability zones. Each zone has its set of pods, secondary storages, providers and network related configuration. Every pod has a bunch of clusters and every cluster a set of hosts and their associated primary storage pools. These configurations are easy to maintain and deploy by just passing them through marvin.

Code Block
bash tsp@cloud:~/cloudstack# nosetests --with-marvin --marvin-config=advanced.cfg -w /tmp #Empty directory where there are no tests to be discovered

Notice that we did not pass the load option to nose. The reason being we want to deploy the configuration specified. This is the default behaviour of Marvin wherein the cloud configuration is deployed and the tests in the directory are run against it. Here we have pointed to a likely empty directory so as to only deploy and configure the zone.

How do I generate it?

The above one host configuration was described as follows:

Code Block

import random
import marvin
from marvin.configGenerator import *

def describeResources():
    zs = cloudstackConfiguration()

    z = zone()
    z.dns1 = '10.147.28.6'
    z.internaldns1 = '10.147.28.6'
    z.name = 'Sandbox-XenServer'
    z.networktype = 'Advanced'
    z.guestcidraddress = '10.1.1.0/24'

    pn = physical_network()
    pn.name = "test-network"
    pn.traffictypes = [traffictype("Guest"), traffictype("Management"), traffictype("Public")]
    z.physical_networks.append(pn)

    p = pod()
    p.name = 'POD0'
    p.gateway = '10.147.29.1'
    p.startip =  '10.147.29.150'
    p.endip =  '10.147.29.159'
    p.netmask = '255.255.255.0'

    v = iprange()
    v.gateway = '10.147.31.1'
    v.startip = '10.147.31.150'
    v.endip = '10.147.31.159'
    v.netmask = '255.255.255.0'
    v.vlan = '31'
    z.ipranges.append(v)

    c = cluster()
    c.clustername = 'C0'
    c.hypervisor = 'XenServer'
    c.clustertype = 'CloudManaged'

    h = host()
    h.username = 'root'
    h.password = 'password'
    h.url = 'http://10.147.29.58'
    c.hosts.append(h)

    ps = primaryStorage()
    ps.name = 'PS0'
    ps.url = 'nfs://10.147.28.6:/export/home/sandbox/primary'
    c.primaryStorages.append(ps)

    p.clusters.append(c)
    z.pods.append(p)

    secondary = secondaryStorage()
    secondary.url = 'nfs://10.147.28.6:/export/home/sandbox/secondary'
    z.secondaryStorages.append(secondary)

    '''Add zone'''
    zs.zones.append(z)

    '''Add mgt server'''
    mgt = managementServer()
    mgt.mgtSvrIp = '10.147.29.111'
    zs.mgtSvr.append(mgt)

    '''Add a database'''
    db = dbServer()
    db.dbSvr = '10.147.29.111'
    db.user = 'cloud'
    db.passwd = 'cloud'
    zs.dbSvr = db

    '''Add some configuration'''
    [zs.globalConfig.append(cfg) for cfg in getGlobalSettings()]

    ''''add loggers'''
    testClientLogger = logger()
    testClientLogger.name = 'TestClient'
    testClientLogger.file = '/var/log/testclient.log'

    testCaseLogger = logger()
    testCaseLogger.name = 'TestCase'
    testCaseLogger.file = '/var/log/testcase.log'

    zs.logger.append(testClientLogger)
    zs.logger.append(testCaseLogger)
    return zs

def getGlobalSettings():
   globals = { "storage.cleanup.interval" : "300",
               "account.cleanup.interval" : "60",
            }

   for k, v in globals.iteritems():
        cfg = configuration()
        cfg.name = k
        cfg.value = v
        yield cfg

if __name__ == '__main__':
    config = describeResources()
    generate_setup_config(config, 'advanced_cloud.cfg')

The zone(), pod(), cluster(), host() are plain objects that carry just attributes. For instance a zone consists of the attributes - name, dns entries, network type etc. Within a zone I create pod()s and append them to my zone object, further down creating cluster()s in those pods and appending them to the pod and within the clusters finally my host()s that get appended to my cluster object. Once I have defined all that is necessary to create my cloud I pass on the described configuration to the generate_setup_config() method which gives me my resultant configuration in JSON format.

Nose Plugins

Nose extends unittest to make testing easier. Nose comes with plugins that help integrating your regular unittests into external build systems, coverage, profiling etc. Marvin comes with its own nose plugin for this so you can use nose to drive CloudStack tests. The plugin is installed on installing marvin. Running nosetests -p will show if the plugin registered successfully.

Code Block

bash
$ nosetests -p
Plugin xunit
Plugin multiprocess
Plugin capture
Plugin logcapture
Plugin coverage
Plugin attributeselector
Plugin doctest
Plugin profile
Plugin collect-only
Plugin isolation
Plugin pdb
Plugin marvin

Nose-timer

There is an interesting plugin named Nose-timer, which helps you to generate the execution time of individual python test. Link here: https://github.com/mahmoudimus/nose-timer

Running nosetest with --with-timer flag.

If you execute testing with marvin, add this parameter in cloud-marvin/pom.xml like this:

Code Block

<executable>nosetests</executable>
<arguments>
  <argument>--with-marvin</argument>
  <argument>--marvin-config</argument>
  <argument>${resolved.user.dir}/${resolved.marvin.config}</argument>
  <argument>--load</argument>
  <argument>-a</argument>
  <argument>tags=${tag}</argument>
  <argument>${resolved.user.dir}/${test}</argument>
  <argument>-v</argument>
  <argument>--with-timer</argument>
</arguments>

Marvin plugin

Code Block
bash $ nosetests --with-marvin --marvin-config=/path/to/basic_zone.cfg --load /path/to/tests

Attribute selection

The smoke tests and component tests contain attributes that can be used to filter the tests that you would like to run against your deployment. You would use nose attrib plugin for this. Following tags are available for filtering:

advanced - Typical Advanced Zone

basic - a basic zone without security groups

sg - a basic zone with security groups

eip - an elastic ip basic zone

advancedns - advanced zone with a netscaler device

advancedsg - advanced zone with security groups

devcloud - tests that will run only for the basic zone on a devcloud setup done using tools/devcloud/devcloud.cfg

speed = 0/1/2 (greater the value lesser the speed)

multihost/multipods/mulitcluster (test requires multiple set of hosts/pods/clusters)

]
}

What you saw in the beginning of the tutorial was a condensed form of this complete configuration file. If you are familiar with the CloudStack installation you will recognize that most of these are settings you give in the install wizards as part of configuration. What is different from the simplified configuration file are the sections zones and globalConfig. The globalConfig section is nothing but a simple listing of (key, value) pairs for the Global Settings section of CloudStack.

The zones section defines the hierarchy of our cloud. At the top-level are the availability zones. Each zone has its set of pods, secondary storages, providers and network related configuration. Every pod has a bunch of clusters and every cluster a set of hosts and their associated primary storage pools. These configurations are easy to maintain and deploy by just passing them through marvin.

Code Block
bash tsp@cloud:~/cloudstack# nosetests --with-marvin --marvin-config=advanced.cfg -deploy -w /tmp #Empty directory where there are no tests to be discovered

Here we have pointed to a likely empty directory so as to only deploy and configure the zone.

How do I generate it?

The above one host configuration was described as follows:

Code Block

import random
import marvin
from marvin.configGenerator import *

def describeResources():
    zs = cloudstackConfiguration()

    z = zone()
    z.dns1 = '10.147.28.6'
    z.internaldns1 = '10.147.28.6'
    z.name = 'Sandbox-XenServer'
    z.networktype = 'Advanced'
    z.guestcidraddress = '10.1.1.0/24'

    pn = physicalNetwork()
    pn.name = "test-network"
    pn.traffictypes = [traffictype("Guest"), traffictype("Management"), traffictype("Public")]
    z.physical_networks.append(pn)

    p = pod()
    p.name = 'POD0'
    p.gateway = '10.147.29.1'
    p.startip =  '10.147.29.150'
    p.endip =  '10.147.29.159'
    p.netmask = '255.255.255.0'

    v = iprange()
    v.gateway = '10.147.31.1'
    v.startip = '10.147.31.150'
    v.endip = '10.147.31.159'
    v.netmask = '255.255.255.0'
    v.vlan = '31'
    z.ipranges.append(v)

    c = cluster()
    c.clustername = 'C0'
    c.hypervisor = 'XenServer'
    c.clustertype = 'CloudManaged'

    h = host()
    h.username = 'root'
    h.password = 'password'
    h.url = 'http://10.147.29.58'
    c.hosts.append(h)

    ps = primaryStorage()
    ps.name = 'PS0'
    ps.url = 'nfs://10.147.28.6:/export/home/sandbox/primary'
    c.primaryStorages.append(ps)

    p.clusters.append(c)
    z.pods.append(p)

    secondary = secondaryStorage()
    secondary.url = 'nfs://10.147.28.6:/export/home/sandbox/secondary'
    z.secondaryStorages.append(secondary)

    '''Add zone'''
    zs.zones.append(z)

    '''Add mgt server'''
    mgt = managementServer()
    mgt.mgtSvrIp = '10.147.29.111'
    zs.mgtSvr.append(mgt)

    '''Add a database'''
    db = dbServer()
    db.dbSvr = '10.147.29.111'
    db.user = 'cloud'
    db.passwd = 'cloud'
    zs.dbSvr = db

    '''Add some configuration'''
    [zs.globalConfig.append(cfg) for cfg in getGlobalSettings()]

    ''''add loggers'''
    testLogger = logger()
    testLogger.logFolderPath = '/tmp/'
    zs.logger = testLogger
	return zs

def getGlobalSettings():
   globals = { "storage.cleanup.interval" : "300",
               "account.cleanup.interval" : "60",
            }

   for k, v in globals.iteritems():
        cfg = configuration()
        cfg.name = k
        cfg.value = v
        yield cfg

if __name__ == '__main__':
    config = describeResources()
    generate_setup_config(config, 'advanced_cloud.cfg')

The zone(), pod(), cluster(), host() are plain objects that carry just attributes. For instance a zone consists of the attributes - name, dns entries, network type etc. Within a zone I create pod()s and append them to my zone object, further down creating cluster()s in those pods and appending them to the pod and within the clusters finally my host()s that get appended to my cluster object. Once I have defined all that is necessary to create my cloud I pass on the described configuration to the generate_setup_config() method which gives me my resultant configuration in JSON format.

Nose Plugins

Nose extends unittest to make testing easier. Nose comes with plugins that help integrating your regular unittests into external build systems, coverage, profiling etc. Marvin comes with its own nose plugin for this so you can use nose to drive CloudStack tests. The plugin is installed on installing marvin. Running nosetests -p will show if the plugin registered successfully.

Code Block

bash
$ nosetests -p
Plugin xunit
Plugin multiprocess
Plugin capture
Plugin logcapture
Plugin coverage
Plugin attributeselector
Plugin doctest
Plugin profile
Plugin collect-only
Plugin isolation
Plugin pdb
Plugin marvin

Nose-timer

There is an interesting plugin named Nose-timer, which helps you to generate the execution time of individual python test. Link here: https://github.com/mahmoudimus/nose-timer

Running nosetest with --with-timer flag.

If you execute testing with marvin, add this parameter in cloud-marvin/pom.xml like this:

Code Block

<executable>nosetests</executable>
<arguments>
  <argument>--with-marvin</argument>
  <argument>--marvin-config</argument>
  <argument>${resolved.user.dir}/${resolved.marvin.config}</argument>
  <argument>-a</argument>
  <argument>tags=${tag}</argument>
  <argument>${resolved.user.dir}/${test}</argument>
  <argument>-v</argument>
  <argument>--with-timer</argument>
</arguments>

Marvin plugin

Code Block
bash $ nosetests --with-marvin --marvin-config=/path/to/basic_zone.cfg /path/to/tests

Attribute selection

The smoke tests and component tests contain attributes that can be used to filter the tests that you would like to run against your deployment. You would use nose attrib plugin for this. Following tags are available for filtering:

advanced - Typical Advanced Zone

basic - a basic zone without security groups

sg - a basic zone with security groups

eip - an elastic ip basic zone

advancedns - advanced zone with a netscaler device

advancedsg - advanced zone with security groups

devcloud - tests that will run only for the basic zone on a devcloud setup done using tools/devcloud/devcloud.cfg

speed = 0/1/2 (greater the value lesser the speed)

multihost/multipods/mulitcluster (test requires multiple set of hosts/pods/clusters)

Code Block

bash
$ nosetests --with-marvin --marvin-config=/path/to/config.cfg -w <test_directory> -a tags=advanced # run tests tagged to run on an advanced zone 

 
#Use below options to run all test cases under smoke directory on advanced zone "and" are provisioning cases, i.e., require hardware to run them. See "hypervisor" option to specify against which hypervisor to run them against, provided your zone and cluster has multiple hosts of various hypervisors type.
$ nosetests-2.7 --with-marvin --marvin-config=/home/abc/softwares/cs_4_4_forward/setup/dev/advanced.cfg  -w /home/abc/softwares/cs_4_4_forward/test/integration/smoke/ --with-xunit --xunit-file=/tmp/bvt_provision_cases.xml --zone=<zone_in_cfg> --hypervisor=<xenserver\kvm\vmware> -a tags=advanced,required_hardware=true


#Use below options to run all test cases under smoke directory on advanced zone "and" are selfservice test cases, i.e., "not" requiring hardware to run them, and can be run against simulator.
$ nosetests-2.7 --with-marvin --marvin-config=/home/abc/softwares/cs_4_4_forward/setup/dev/advanced.cfg  -w /home/abc/softwares/cs_4_4_forward/test/integration/smoke/ --with-xunit --xunit-file=/tmp/bvt_selfservice_cases.xml --zone=<zone_in_cfg> --hypervisor=simulator -a tags=advanced,required_hardware=false
 
#Use below options to run all test cases under smoke directory on advanced zone "and" are selfservice test cases, i.e., "not" requiring hardware to run them, and can be run against simulator. As well, below "deploy" option takes care to deploy datacenter as well.
$ nosetests-2.7 --with-marvin --marvin-config=/home/abc/softwares/cs_4_4_forward/setup/dev/advanced.cfg  -w /home/abc/softwares/cs_4_4_forward/test/integration/smoke/ --with-xunit --xunit-file=/tmp/bvt_provision_cases.xml --zone=<zone_in_cfg> --hypervisor=simulator -a tags=advanced,required_hardware=false --deploy

Code Block

bash
$ nosetests --with-marvin --marvin-config=/path/to/config.cfg -w test_directory -a tags=advanced,tags=simulator # run tests tagged to run on an advanced zone with the simulator

Guidelines to choose scenarios for integration

...

Space shortcuts

Child pages

Versions Compared

Old Version 73

New Version Current

Key

Windows specifics

Installation

Parts of the test

imports

Parts of the test

imports

TestDeployVM

Test Categories

Running the test

IDE - Eclipse and PyDev

Checkin/smoke Tests

Run the Tests

Including your own

How do I generate it?

Nose Plugins

Nose-timer

Marvin plugin

Attribute selection

How do I generate it?

Nose Plugins

Nose-timer

Marvin plugin

Attribute selection

Guidelines to choose scenarios for integration