Page History

...

1. Checkout the incubator-cloudstack project from incubator-cloudstack.git
2. You will need Python - version 2.6 to install marvin but 2.7 to run the tests. Additional modules are installed automatically by Marvin - python-paramiko, mysql-connector-python, nose, should_dsl, marvin-nose.
3. For Windows development environment, following additional steps are needed to be able to run the python tests or for using easy_install via cygwin:

• Update or Install Cygwin with Python 2.7 package. If you use a separate python then it fails to find files since it cannot look up the cygwin style paths. Use this python while compiling the developer profile via maven.
• The mysql-connector-python for Windows can be found here.
• To be able to use easy_install on Windows, download it from here https://pypi.python.org/pypi/setuptools OR follow this http://www.vertigrated.com/blog/2010/08/installing-easy_install-on-windows-7-using-python-2-7/. Ideally you should install pip to be able to do automatic sync of marvin packages (see marvin.synchttps://cwiki.apache.org/CLOUDSTACK/how-to-build-on-master-branch.html#Howtobuildonmasterbranch-SynclatestAPIs)

1. Eclipse is assumed to be your development environment. Install the PyDev plugin by following the manualhttp://pydev.org/manual_101_root.htmlYou should install Eclipse and the PyDev plugin. PyDev features auto-completion for python modules from within the Eclipse environment.
2. On the master branch the 'developer' profile compiles , and packages and installs Marvin. It does NOT install to your default PYTHONPATH.

   Code Block
   mvn -P developer -pl :cloud-marvin

3. You may will install Marvin using the tarball by hand by . Use pip or easy_install. pip is recommended

   Code Block
   pip install tools/marvin/dist/Marvin-0.1.0.tar.gz

easy_install tools/marvin/dist/Marvin-0.1.0.tar.gz

• For Windows developers, it will be: easy_install tools/marvin/dist/Marvin-0.1.0.zipwindows developers - This may fail while installing the pycrypto dependency since gcc/glibc are required which are not available by default. If it does, just install pycrypto binaries from binary available from here http://www.voidspace.org.uk/python/modules.shtml#pycrypto and re-run easy_install.

...

If you are a QA engineer you won't need the entire codebase to build work with marvin and write tests.

1. Jenkins@builds.a.o holds artifacts of the marvin builds and you can download itthe latest artifact.
2. The artifact (.tar.gz) is available after the build succeeds. Download it.
3. On the client machine where you will be writing/running tests from - setup the following:
   1. Install python 2.7 (http://www.python.org/download/releases/)
   2. Install setuptools. Follow the instructions for your client machine (windows/linux/mac)
   3. (for Windows only) Install pycrypto on Windows because windows does not bundle gcc to compile pycrypto. 
4. The Marvin artifact you downloaded can now be installed. Any required python packages will be installed automatically

   Code Block
   easy_install tools/marvin/dist/Marvin-0.1.0.tar.gz

5. To test if the installation was successful get into a python shell

   Code Block
   root@cloud:~/cloudstack-oss/tools/marvin/dist# python Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) [GCC 4.5.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import marvin >>> from marvin.cloudstackAPI import *

   imports should happen without reporting errors.

First Steps

_To jump into test case writing move on to section _

In our first steps we will build a simple API call and fire it against a CloudStack management server that is already deployed, configured and ready to accept API calls. You can pick any management server in your lab that has a few VMs running on it or you can use DevCloud. Create a sample json config file telling us where your management server and database server are. Here's a sample:

prasanna@cloud:~cloudstack-oss# cat demo/demo.cfg
{
    "dbSvr": {
        "dbSvr": "automation.lab.vmops.com",
        "passwd": "cloud",
        "db": "cloud",
        "port": 3306,
        "user": "cloud"
    },
    "logger": [
        {
            "name": "TestClient",
            "file": "/var/log/testclient.log"
        },
        {
            "name": "TestCase",
            "file": "/var/log/testcase.log"
        }
    ],
    "mgtSvr": [
        {
            "mgtSvrIp": "automationmarvin.test-lab.vmops.com",
            "port": 8096
        }
    ]
}

• Note: dbSvr is the location where mysql server is running and passwd is the password for user cloud.
• Run this command to open up the iptables integration.port on your management server iptables -I INPUT -p tcp --dport 8096 -j ACCEPT
• Change the global setting integration.api.port on the CloudStack GUI to 8096 and restart the management server.

1. Enter an interactive python shell and follow along with the steps listed below. We've used the ipython shell in our example because it has a very handy auto-complete feature for python libraries.
2. We will import a few essential libraries to start with.
   • The cloudstackTestCase module contains the essential API calls we need and a reference to the API client itself. All tests will be children (subclasses) of the cloudstackTestCase since it contains the toolkit (attributes) to do our testing

     Code Block
     In [1]: import marvin In [2]: from marvin.cloudstackTestCase import *

   • The deployDataCenter module imported below will help us load our json configuration file we wrote down in the beginning so we can tell the test framework that we have our management server configured and ready

     Code Block
     In [2]: import marvin.deployDataCenter

3. Let's load the configuration file using the deployDataCenter module

   Code Block
   In [3]: config = marvin.deployDataCenter.deployDataCenters('demo/demo.cfg') In [4]: config.loadCfg()

4. Once the configuration is loaded successfully, all we'll need is an instance of the apiClient which will help fire our cloudstack APIs against the configured management server. In addition to the apiClient, the test framework also provides a dbClient to help us fire any SQL queries against the database for verification. So let's go ahead and get a reference to the apiClient:

   Code Block
   In [5]: apiClient = config.testClient.getApiClient()

5. Now we'll start with forming a very simple API call. listConfigurations - which will show us the "global settings" that are set on our instance of CloudStack. The API command is instantiated as shown in the code snippet (as are other API commands).

   Code Block
   In [6]: listconfig = listConfigurations.listConfigurationsCmd()

   So the framework is intuitive in the verbs used for an API call. To deploy a VM you would be calling the deployVirtualMachineCmd method inside the deployVirtualMachine object. Simple, ain't it?
6. Since it's a large list of global configurations let's limit ourselves to fetch only the configuration with the keyword 'expunge'. Let's change our listconfig object to take this attribute as follows:

   Code Block
   In [7]: listconfig.name = 'expunge'

7. And finally - we fire the call using the apiClient as shown below:

   Code Block
   In [8]: listconfigresponse = apiClient.listConfigurations(listconfig)

   Lo' and Behold - the response you've awaited:

   Code Block

   In [9]: print listconfigresponse [ {category : u'Advanced', name : u'expunge.delay', value : u'60', description : u'Determines how long (in seconds) to wait before actually expunging destroyed vm. The default value = the default value of expunge.interval'}, {category : u'Advanced', name : u'expunge.interval', value : u'60', description : u'The interval (in seconds) to wait before running the expunge thread.'}, {category : u'Advanced', name : u'expunge.workers', value : u'3', description : u'Number of workers performing expunge '}]

...

Listing stuff is all fine and dandy you might say - How do I launch VMs using python? And do I use the shell each time I have to do this? Well clearly not, we can have all the steps compressed into a python script. This example will show such a script marvin and the cloudstack API? This following example will show working with python's unittest which will:

• create a testcase class
• setUp a user account - name: user , passwd: password
• deploy a VM into that user account using the default small service offering and CentOS template
• verify that the VM we deployed reached the 'Running' state
• tearDown the user account - basically delete it to cleanup acquired resources

Without much ado, here's the script:

#!/usr/bin/env python

import marvin
from marvin import cloudstackTestCase
from marvin.cloudstackTestCase import *

import unittest
import hashlib
import random

class TestDeployVm(cloudstackTestCase):
    """
    This test deploys a virtual machine into a user account
    using the small service offering and builtin template
    """
    def setUp(self):
        """
        CloudStack internally saves its passwords in md5 form and that is how we
        specify it in the API. Python's hashlib library helps us to quickly hash
        strings as follows
        """
        mdf = hashlib.md5()
        mdf.update('password')
        mdf_pass = mdf.hexdigest()

/env python

import marvin
from marvin import cloudstackTestCase
from marvin.cloudstackTestCase import *

class TestDeployVm(cloudstackTestCase):
    """Test deploying a virtual machine into a user account
    """
    def setUp(self):
        self.apiClient = self.testClient.getApiClient() #Get ourselves an API client

        self.acct = createAccount.createAccountCmd() #The createAccount command
        self.acct.accounttype = 0                    #We need a regular user. admins have accounttype=1
        self.acct.firstname = 'bugs'
        self.acct.lastname = 'bunny'                 #What's up doc?
        self.acct.password = mdf_pass      'password'          #The md5 hashed password string
        self.acct.username = 'bugs'
        self.acct.email = 'bugs@rabbithole.com'
        self.acct.account = 'bugs'
        self.acct.domainid = 1                       #The default ROOT domain
        self.acctResponse = self.apiClient.createAccount(self.acct)
        # And upon successful creation we'll log a helpful message in our logs
        # using the default debug logger of the test framework
        self.debug("successfully created account: %s, user: %s, id: \
                   %s"%(self.acctResponse.account.account, \
                        self.acctResponse.account.username, \
                        self.acctResponse.account.id))

    def test_DeployVm(self):
        """
        Let's start by defining the attributes of our VM that we will be
        deploying on CloudStack. We will be assuming a single zone is available
        and is configured and all templates are Ready

        The hardcoded values are used only for brevity.
        """
        deployVmCmd = deployVirtualMachine.deployVirtualMachineCmd()
        deployVmCmd.zoneid = 1
        deployVmCmd.account = self.acct.account
        deployVmCmd.domainid = self.acct.domainid
        deployVmCmd.templateid = 5                   #For default template- CentOS 5.6(64 bit)
        deployVmCmd.serviceofferingid = 1

        deployVmResponse = self.apiClient.deployVirtualMachine(deployVmCmd)
        self.debug("VM %s was deployed in the job %s"%(deployVmResponse.id, deployVmResponse.jobid))

        # At this point our VM is expected to be Running. Let's find out what
        # listVirtualMachines tells us about VMs in this account

        listVmCmd = listVirtualMachines.listVirtualMachinesCmd()
        listVmCmd.id = deployVmResponse.id
        listVmResponse = self.apiClient.listVirtualMachines(listVmCmd)

        self.assertNotEqual(len(listVmResponse), 0, "Check if the list API \
                            returns a non-empty response")

        vm = listVmResponse[0]

        self.assertEqual(vm.id, deployVmResponse.id, "Check if the VM returned \
                         is the same as the one we deployed")


        self.assertEqual(vm.state, "Running", "Check if VM has reached \
                         a state of running")

    def tearDown(self):                               # Teardown will delete the Account as well as the VM once the VM reaches "Running" state
        """
        And finally let us cleanup the resources we created by deleting the
        account. All good unittests are atomic and rerunnable this way
        """
        deleteAcct = deleteAccount.deleteAccountCmd()
        deleteAcct.id = self.acctResponse.account.id
        self.apiClient.deleteAccount(deleteAcct)

...