Page History

...

Checkout the cloudstack-oss project from git:incubator.apache.org
You will need Python - version 2.7. Additional modules - python-paramiko, ipython (optional but useful), nose (optional)
You should install Eclipse and the PyDev plugin. PyDev features auto-completion and some good documentation on how to install it. Since the Dev team is already using Eclipse, this is a good option to consider to speed up writing your tests.
There is an ant target (package-marvin) that you can run which will create a source tarball in tools/marvin/dist
On the master branch the 'developer' profile compiles, packages and installs Marvin.
Code Block
mvn -P developer
The mvn deploy goal will install marvin using pip. If not you may install it by hand
Code Block
pip install tools/marvin/dist/
Install this tarball with pip (pip install
Marvin-0.1.0.tar.gz
)
Reference: The Python testing framework for more details on the organization of the test framework.

QA

If you are a QA engineer

Jenkins holds artifacts of the marvin builds and you can download the latest hereit.
The artifact (.tar.gz) is available after the build succeeds. Download it.
On the client machine where you will be writing/running tests from setup the following:
1. Install python 2.7 (http://www.python.org/download/releases/^{Image Removed})
2. Install setuptools. Follow the instructions for your client machine from here
3. Install pip. http://www.pip-installer.org/en/latest/installing.html^{Image Removed}
The Marvin artifact you downloaded can now be installed using pip. Any required python packages will be installed automatically
1. pip install Marvin-.*.tar.gz

To test if the installation was successful get into a python shell

Code Block

root@cloud:~/cloudstack-oss/tools/marvin/dist# python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import marvin
>>>

import should happen without reporting errors.

...

Code Block

title	JSON configuration

prasanna@cloud:~cloudstack-oss# cat demo/demo.cfg
{
    "dbSvr": {
        "dbSvr": "automation.lab.vmops.com", 
        "passwd": "cloud",
        "db": "cloud", 
        "port": 3306, 
        "user": "cloud"
    }, 
    "logger": [
        {
            "name": "TestClient", 
            "file": "/var/log/testclient.log"
        }, 
        {
            "name": "TestCase", 
            "file": "/var/log/testcase.log"
        }
    ], 
    "mgtSvr": [
        {
            "mgtSvrIp": "automation.lab.vmops.com", 
            "port": 8096
        }
    ]
}

Note: dbSvr is the location where mysql server is running and passwd is the password for user cloud.
Run this command to open up the iptables on your management server iptables -I INPUT -p tcp --dport 8096 -j ACCEPT
Change the global setting integration.api.port on the CloudStack GUI to 8096 and restart the management server.

Enter an interactive python shell and follow along with the steps listed below. We've used the ipython shell in our example because it has a very handy auto-complete feature
We will import a few essential libraries to start with.
- The cloudstackTestCase module contains the essential API calls we need and a reference to the API client itself. All tests will be children (subclasses) of the cloudstackTestCase since it contains the toolkit (attributes) to do our testing
  Code Block
  In [1]: import marvin In [2]: from marvin.cloudstackTestCase import *
- The deployDataCenter module imported below will help us load our json configuration file we wrote down in the beginning so we can tell the test framework that we have our management server configured and ready
  Code Block
  In [2]: import marvin.deployDataCenter

Let's load the configuration file using the deployDataCenter module

Code Block
In [3]: config = marvin.deployDataCenter.deployDataCenters('demo/demo.cfg') In [4]: config.loadCfg()

Once the configuration is loaded successfully, all we'll need is an instance of the apiClient which will help fire our cloudstack APIs against the configured management server. In addition to the apiClient, the test framework also provides a dbClient to help us fire any SQL queries against the database for verification. So let's go ahead and get a reference to the apiClient:
Code Block
In [5]: apiClient = config.testClient.getApiClient()
Now we'll start with forming a very simple API call. listConfigurations - which will show us the "global settings" that are set on our instance of CloudStack. The API command is instantiated as shown in the code snippet (as are other API commands).
Code Block
In [6]: listconfig = listConfigurations.listConfigurationsCmd()
So the framework is intuitive in the verbs used for an API call. To deploy a VM you would be calling the deployVirtualMachineCmd method inside the deployVirtualMachine object. Simple, ain't it?
Since it's a large list of global configurations let's limit ourselves to fetch only the configuration with the keyword 'expunge'. Let's change our listconfig object to take this attribute as follows:
Code Block
In [7]: listconfig.name = 'expunge'

And finally - we fire the call using the apiClient as shown below:

Code Block
In [8]: listconfigresponse = apiClient.listConfigurations(listconfig)

Lo' and Behold - the response you've awaited:

Code Block

In [9]: print listconfigresponse
 
[ {category : u'Advanced', name : u'expunge.delay', value : u'60', description : u'Determines how long (in seconds) to wait before actually expunging destroyed vm. The default value = the default value of expunge.interval'},
  {category : u'Advanced', name : u'expunge.interval', value : u'60', description : u'The interval (in seconds) to wait before running the expunge thread.'},
  {category : u'Advanced', name : u'expunge.workers', value : u'3', description : u'Number of workers performing expunge '}]

...

Code Block

#!/usr/bin/env python

import marvin
from marvin import cloudstackTestCase
from marvin.cloudstackTestCase import *

import unittest
import hashlib
import random

class TestDeployVm(cloudstackTestCase):
    """
    This test deploys a virtual machine into a user account 
    using the small service offering and builtin template
    """
    def setUp(self):
        """
        CloudStack internally saves its passwords in md5 form and that is how we
        specify it in the API. Python's hashlib library helps us to quickly hash
        strings as follows
        """
        mdf = hashlib.md5()
        mdf.update('password')
        mdf_pass = mdf.hexdigest()

        self.apiClient = self.testClient.getApiClient() #Get ourselves an API client

        self.acct = createAccount.createAccountCmd() #The createAccount command
        self.acct.accounttype = 0                    #We need a regular user. admins have accounttype=1
        self.acct.firstname = 'bugs'                 
        self.acct.lastname = 'bunny'                 #What's up doc?
        self.acct.password = mdf_pass                #The md5 hashed password string
        self.acct.username = 'bugs'
        self.acct.email = 'bugs@rabbithole.com'
        self.acct.account = 'bugs'
        self.acct.domainid = 1                       #The default ROOT domain
        self.acctResponse = self.apiClient.createAccount(self.acct)
        # And upon successful creation we'll log a helpful message in our logs
        # using the default debug logger of the test framework
        self.debug("successfully created account: %s, user: %s, id: \
                   %s"%(self.acctResponse.account.account, \
                        self.acctResponse.account.username, \
                        self.acctResponse.account.id))

    def test_DeployVm(self):
        """
        Let's start by defining the attributes of our VM that we will be
        deploying on CloudStack. We will be assuming a single zone is available
        and is configured and all templates are Ready

        The hardcoded values are used only for brevity. 
        """
        deployVmCmd = deployVirtualMachine.deployVirtualMachineCmd()
        deployVmCmd.zoneid = 1
        deployVmCmd.account = self.acct.account
        deployVmCmd.domainid = self.acct.domainid
        deployVmCmd.templateid = 5                   #For default template- CentOS 5.6(64 bit)
        deployVmCmd.serviceofferingid = 1

        deployVmResponse = self.apiClient.deployVirtualMachine(deployVmCmd)
        self.debug("VM %s was deployed in the job %s"%(deployVmResponse.id, deployVmResponse.jobid))

        # At this point our VM is expected to be Running. Let's find out what
        # listVirtualMachines tells us about VMs in this account

        listVmCmd = listVirtualMachines.listVirtualMachinesCmd()
        listVmCmd.id = deployVmResponse.id
        listVmResponse = self.apiClient.listVirtualMachines(listVmCmd)

        self.assertNotEqual(len(listVmResponse), 0, "Check if the list API \
                            returns a non-empty response")

        vm = listVmResponse[0]

        self.assertEqual(vm.id, deployVmResponse.id, "Check if the VM returned \
                         is the same as the one we deployed")


        self.assertEqual(vm.state, "Running", "Check if VM has reached \
                         a state of running")

    def tearDown(self):                               # Teardown will delete the Account as well as the VM once the VM reaches "Running" state
        """
        And finally let us cleanup the resources we created by deleting the
        account. All good unittests are atomic and rerunnable this way
        """
        deleteAcct = deleteAccount.deleteAccountCmd()
        deleteAcct.id = self.acctResponse.account.id
        self.apiClient.deleteAccount(deleteAcct)

...

Code Block

root@cloud:~/cloudstack-oss# python -m marvin.deployAndRun -c demo/demo.cfg -t /tmp/testcase.log -r /tmp/results.log -f demo/TestDeployVm.py -l

root@cloud:~/cloudstack-oss# cat /tmp/results.log 
test_DeployVm (testDeployVM.TestDeployVm) ... ok
----------------------------------------------------------------------
Ran 1 test in 100.511s
OK

...

Code Block

#!/usr/bin/env python

import marvin
from marvin import cloudstackTestCase
from marvin.cloudstackTestCase import *
from marvin.remoteSSHClient import remoteSSHClient 

import unittest
import hashlib
import random
import string

class TestSshDeployVm(cloudstackTestCase):
    """
    This test deploys a virtual machine into a user account 
    using the small service offering and builtin template
    """
    @classmethod
    def setUpClass(cls):
        """
        CloudStack internally saves its passwords in md5 form and that is how we
        specify it in the API. Python's hashlib library helps us to quickly hash
        strings as follows
        """
        mdf = hashlib.md5()
        mdf.update('password')
        mdf_pass = mdf.hexdigest()
        acctName = 'bugs-'+''.join(random.choice(string.ascii_uppercase + string.digits) for x in range(6)) #randomly generated account

        cls.apiClient = super(TestSshDeployVm, cls).getClsTestClient().getApiClient()  
        cls.acct = createAccount.createAccountCmd() #The createAccount command
        cls.acct.accounttype = 0                    #We need a regular user. admins have accounttype=1
        cls.acct.firstname = 'bugs'                 
        cls.acct.lastname = 'bunny'                 #What's up doc?
        cls.acct.password = mdf_pass                #The md5 hashed password string
        cls.acct.username = acctName
        cls.acct.email = 'bugs@rabbithole.com'
        cls.acct.account = acctName
        cls.acct.domainid = 1                       #The default ROOT domain
        cls.acctResponse = cls.apiClient.createAccount(cls.acct)
        
    def setUpNAT(self, virtualmachineid):
        listSourceNat = listPublicIpAddresses.listPublicIpAddressesCmd()
        listSourceNat.account = self.acct.account
        listSourceNat.domainid = self.acct.domainid
        listSourceNat.issourcenat = True
        
        listsnatresponse = self.apiClient.listPublicIpAddresses(listSourceNat)
        self.assertNotEqual(len(listsnatresponse), 0, "Found a source NAT for the acct %s"%self.acct.account)
        
        snatid = listsnatresponse[0].id
        snatip = listsnatresponse[0].ipaddress
        
        try:
            createFwRule = createFirewallRule.createFirewallRuleCmd()
            createFwRule.cidrlist = "0.0.0.0/0"
            createFwRule.startport = 22
            createFwRule.endport = 22
            createFwRule.ipaddressid = snatid
            createFwRule.protocol = "tcp"
            createfwresponse = self.apiClient.createFirewallRule(createFwRule)
            
            createPfRule = createPortForwardingRule.createPortForwardingRuleCmd()
            createPfRule.privateport = 22
            createPfRule.publicport = 22
            createPfRule.virtualmachineid = virtualmachineid
            createPfRule.ipaddressid = snatid
            createPfRule.protocol = "tcp"
            
            createpfresponse = self.apiClient.createPortForwardingRule(createPfRule)
        except e:
            self.debug("Failed to create PF rule in account %s due to %s"%(self.acct.account, e))
            raise e
        finally:
            return snatip        

    def test_SshDeployVm(self):
        """
        Let's start by defining the attributes of our VM that we will be
        deploying on CloudStack. We will be assuming a single zone is available
        and is configured and all templates are Ready

        The hardcoded values are used only for brevity. 
        """
        deployVmCmd = deployVirtualMachine.deployVirtualMachineCmd()
        deployVmCmd.zoneid = 1
        deployVmCmd.account = self.acct.account
        deployVmCmd.domainid = self.acct.domainid
        deployVmCmd.templateid = 5 #CentOS 5.6 builtin
        deployVmCmd.serviceofferingid = 1

        deployVmResponse = self.apiClient.deployVirtualMachine(deployVmCmd)
        self.debug("VM %s was deployed in the job %s"%(deployVmResponse.id, deployVmResponse.jobid))

        # At this point our VM is expected to be Running. Let's find out what
        # listVirtualMachines tells us about VMs in this account

        listVmCmd = listVirtualMachines.listVirtualMachinesCmd()
        listVmCmd.id = deployVmResponse.id
        listVmResponse = self.apiClient.listVirtualMachines(listVmCmd)

        self.assertNotEqual(len(listVmResponse), 0, "Check if the list API \
                            returns a non-empty response")

        vm = listVmResponse[0]
        hostname = vm.name
        nattedip = self.setUpNAT(vm.id)

        self.assertEqual(vm.id, deployVmResponse.id, "Check if the VM returned \
                         is the same as the one we deployed")


        self.assertEqual(vm.state, "Running", "Check if VM has reached \
                         a state of running")

        # SSH login and compare hostname        
        ssh_client = remoteSSHClient(nattedip, 22, "root", "password")
        stdout = ssh_client.execute("hostname")
        
        self.assertEqual(hostname, stdout[0], "cloudstack VM name and hostname match")


    @classmethod
    def tearDownClass(cls):
        """
        And finally let us cleanup the resources we created by deleting the
        account. All good unittests are atomic and rerunnable this way
        """
        deleteAcct = deleteAccount.deleteAccountCmd()
        deleteAcct.id = cls.acctResponse.account.id
        cls.apiClient.deleteAccount(deleteAcct)

...

Code Block

{
    "zones": [
        {
            "name": "Sandbox-XenServer", 
            "guestcidraddress": "10.1.1.0/24", 
            "physical_networks": [
                {
                    "broadcastdomainrange": "Zone", 
                    "name": "test-network", 
                    "traffictypes": [
                        {
                            "typ": "Guest"
                        }, 
                        {
                            "typ": "Management"
                        }, 
                        {
                            "typ": "Public"
                        }
                    ], 
                    "providers": [
                        {
                            "broadcastdomainrange": "ZONE", 
                            "name": "VirtualRouter"
                        }
                    ]
                }
            ], 
            "dns1": "10.147.28.6", 
            "ipranges": [
                {
                    "startip": "10.147.31.150", 
                    "endip": "10.147.31.159", 
                    "netmask": "255.255.255.0", 
                    "vlan": "31", 
                    "gateway": "10.147.31.1"
                }
            ], 
            "networktype": "Advanced", 
            "pods": [
                {
                    "endip": "10.147.29.159", 
                    "name": "POD0", 
                    "startip": "10.147.29.150", 
                    "netmask": "255.255.255.0", 
                    "clusters": [
                        {
                            "clustername": "C0", 
                            "hypervisor": "XenServer", 
                            "hosts": [
                                {
                                    "username": "root", 
                                    "url": "http://10.147.29.58", 
                                    "password": "password"
                                }
                            ], 
                            "clustertype": "CloudManaged", 
                            "primaryStorages": [
                                {
                                    "url": "nfs://10.147.28.6:/export/home/sandbox/primary", 
                                    "name": "PS0"
                                }
                            ]
                        }
                    ], 
                    "gateway": "10.147.29.1"
                }
            ], 
            "internaldns1": "10.147.28.6", 
            "secondaryStorages": [
                {
                    "url": "nfs://10.147.28.6:/export/home/sandbox/secondary"
                }
            ]
        }
    ], 
    "dbSvr": {
        "dbSvr": "10.147.29.111", 
        "passwd": "cloud", 
        "db": "cloud", 
        "port": 3306, 
        "user": "cloud"
    }, 
    "logger": [
        {
            "name": "TestClient", 
            "file": "/var/log/testclient.log"
        }, 
        {
            "name": "TestCase", 
            "file": "/var/log/testcase.log"
        }
    ], 
    "globalConfig": [
        {
            "name": "storage.cleanup.interval", 
            "value": "300"
        }, 
        {
            "name": "account.cleanup.interval", 
            "value": "600"
        }
    ], 
    "mgtSvr": [
        {
            "mgtSvrIp": "10.147.29.111", 
            "port": 8096
        }
    ]
}

What you saw earlier was a condensed form of this complete configuration file. If you're familiar with the CloudStack installation you will recognize that most of these are settings you give in the install wizards as part of configuration. What is different from the simplified configuration file are the sections "zones" and "globalConfig". The globalConfig section is nothing but a simple listing of (key, value) pairs for the "Global Settings" section of CloudStack.

...

Code Block

#!/usr/bin/env python

import random
import marvin
from marvin.configGenerator import *

def describeResources():
    zs = cloudstackConfiguration()

    z = zone()
    z.dns1 = '10.147.28.6'
    z.internaldns1 = '10.147.28.6'
    z.name = 'Sandbox-XenServer'
    z.networktype = 'Advanced'
    z.guestcidraddress = '10.1.1.0/24'
 
    pn = physical_network()
    pn.name = "test-network"
    pn.traffictypes = [traffictype("Guest"), traffictype("Management"), traffictype("Public")]
    z.physical_networks.append(pn)

    p = pod()
    p.name = 'POD0'
    p.gateway = '10.147.29.1'
    p.startip =  '10.147.29.150'
    p.endip =  '10.147.29.159'
    p.netmask = '255.255.255.0'

    v = iprange()
    v.gateway = '10.147.31.1'
    v.startip = '10.147.31.150'
    v.endip = '10.147.31.159'
    v.netmask = '255.255.255.0'
    v.vlan = '31'
    z.ipranges.append(v)

    c = cluster()
    c.clustername = 'C0'
    c.hypervisor = 'XenServer'
    c.clustertype = 'CloudManaged'

    h = host()
    h.username = 'root'
    h.password = 'password'
    h.url = 'http://10.147.29.58'
    c.hosts.append(h)

    ps = primaryStorage()
    ps.name = 'PS0'
    ps.url = 'nfs://10.147.28.6:/export/home/sandbox/primary'
    c.primaryStorages.append(ps)

    p.clusters.append(c)
    z.pods.append(p)

    secondary = secondaryStorage()
    secondary.url = 'nfs://10.147.28.6:/export/home/sandbox/secondary'
    z.secondaryStorages.append(secondary)

    '''Add zone'''
    zs.zones.append(z)

    '''Add mgt server'''
    mgt = managementServer()
    mgt.mgtSvrIp = '10.147.29.111'
    zs.mgtSvr.append(mgt)

    '''Add a database'''
    db = dbServer()
    db.dbSvr = '10.147.29.111'
    db.user = 'cloud'
    db.passwd = 'cloud'
    zs.dbSvr = db

    '''Add some configuration'''
    [zs.globalConfig.append(cfg) for cfg in getGlobalSettings()]

    ''''add loggers'''
    testClientLogger = logger()
    testClientLogger.name = 'TestClient'
    testClientLogger.file = '/var/log/testclient.log'

    testCaseLogger = logger()
    testCaseLogger.name = 'TestCase'
    testCaseLogger.file = '/var/log/testcase.log'

    zs.logger.append(testClientLogger)
    zs.logger.append(testCaseLogger)
    return zs

def getGlobalSettings():
   globals = [
        {
            "name": "storage.cleanup.interval", 
            "value": "300"
        }, 
        {
            "name": "account.cleanup.interval", 
            "value": "600"
        }
    ]

   for k, v in globals:
        cfg = configuration()
        cfg.name = k
        cfg.value = v
        yield cfg

if __name__ == '__main__':
    config = describeResources()
    generate_setup_config(config, 'advanced_cloud.cfg')

...

Code Block

borderStyle	solid

root@cloud:~/incubator-cloudstack/tools/marvin/marvin/sandbox/advanced# python advanced_env.py -i setup.properties -o advanced.cfg
root@cloud:~/incubator-cloudstack/tools/marvin/marvin/sandbox/advanced# head -10 advanced.cfg
{
    "zones": [
        {
            "name": "Sandbox-XenServer",
            "guestcidraddress": "10.1.1.0/24",
            
... <snip/> ...

Marvin Nose Plugin

...

Code Block

borderStyle	solid

$ cd /usr/local/lib/python2.7/site-packages/marvin
$ easy_install .
Processing .
Running setup.py -q bdist_egg --dist-dir 
Installed /usr/local/lib/python2.7/dist-packages/marvin_nose-0.1.0-py2.7.egg
Processing dependencies for marvin-nose==0.1.0
Finished processing dependencies for marvin-nose==0.1.0
 
$ nosetests -p
Plugin xunit
Plugin multiprocess
Plugin capture
Plugin logcapture
Plugin coverage
Plugin attributeselector
Plugin doctest
Plugin profile
Plugin collect-only
Plugin isolation
Plugin pdb
Plugin marvin


# Usage and running tests
$ nosetests --with-marvin --marvin-config=/path/to/basic_zone.cfg --load /path/to/tests

...

The single largest python resource is the python website itself - http://www.python.org^{Image Removed}
Mark Pilgrim's - "Dive Into Python" - is another great resource. The book is available for free online - http://www.diveintopython.net^{Image Removed}. Chapter 1- 6 cover a good portion of language basics and Chapter 13 & 14 are essential for anyone doing test script development
To read more about the assert methods the language reference is the ideal place - http://docs.python.org/library/unittest.html^{Image Removed}
Python testing cookbook: We have a local copy that can be shared for internal use only. Please email me.

...

Space shortcuts

Child pages

Versions Compared

Old Version 22

New Version 23

Key

QA

Marvin Nose Plugin