Page History

...

22. Use pylint, tabnanny and other code checker tools to run through the test code before commit. Follow pep8 and known coding standards as below both for naming convention and other standards. http://www.python.org/dev/peps/pep-0008/ http://google-styleguide.googlecode.com/svn/trunk/pyguide.html

23. Raise all exceptions including generic ones through CS related exception module, this way all exceptions for debug purpose can be logged or tracked through one single module, don’t write separate exception classes for each test.

24. Instead of adding multiple lines of asserts as below to the test code, Please use a simple "verifyElementInList" utility function added to cs/tools/integration/lib/utils.py. See the usage below.

self.assertEqual(
isinstance(list_network_offerings_response, list),
True,
"listNetworkOfferings returned invalid object in response."
)

self.assertNotEqual(
len(list_network_offerings_response),
0,
"listNetworkOfferings returned empty list."
)

self.assertEqual(
list_network_offerings_response0.state,
"Enabled",
"The network offering state should get updated to Enabled."
)

If we want to verify whether "a" is a list, its empty or not, then verify element 1 is available in the list and assert, then the below line ( generic example, change it to whichever assert we want ) should suffice.

assert verifyElementInList(a,1)0,PASS,"TC FAILED"

We just need to import verifyElementInList from utils before using it

25. Please raise a review issue if there are no module level document strings,or wherever we see relevant documentation is missing.

26. Remove hard coded string values EX: values for ip addresses, urls, hostnames, and other configuration information EX: Services class from test feature code. Instead, separate it in a config file ( If test feature is named test_abc.py, then follow naming convention for feature configuration as test_abc.cfg ), place this configuration file to config folder under cs/tools/marvin/marvin. Once checked in, the new configuration is then available to test feature as a dictionary for usage by using ConfigManager Interface. Please use the new ConfigManager class support added under marvin/marvin/configGeneratory.py for this. See the below example.

'''
1. cloudstackTestClient will now have configuration object exposed
and available through its method getConfigParser()
2. The object will now have all options exposed through ConfigManager
3. cls.configObj will now provide ConfigManager interface
User can use this to parse and get the dictionary represenation
of his configuaration
'''
@classmethod
def setUpClass(cls):
cls.api_client = super(
TestAddMultipleNetScaler,
cls
).getClsTestClient().getApiClient()
cls.configObj = super(
TestAddMultipleNetScaler,
cls
).getConfigParser()
if cls.configObj is not None:
'''
Here, cls.configDict will now have all the configuration parsed and available as Dictionary
This Dictionary is available to total test feature code accessible wherever applicable
'''
cls.configDict = cls.configObj.getConfig(self, "/setup/config/myfeature.cfg", None )

Now, cls.configDict will have dictionary representation for your configuration and we can use it through out the code.

27. For better readability and maintainability, maintain all widely and more often used codes under codes.py ( new module added under cs/tools/marvin/marvin/codes.py ). See codes.py for specific example.

EX: Replace "Diabled","Enabled","network_offering" strings etc in test code, with readable and well named constants and notations, place them under codes.py. Then, use them uniformly across the code for test features, with these we need alteration at one place if required and change gets available to all features. We as well avoid different naming notations under different test features.

28. Use uniform naming and usage of object creation wherever possible. See the below example current usage of creating apiclient ( cls.api_client and self.apiclient ) as such both are same objects but the way, creation is made different. Instead ,we can use common way of creating and retrieving apiclient, use the way we created api_client under setUpClass as well for setUp method we want.

class TestPublicIP(cloudstackTestCase):

def setUp(self):
self.apiclient = self.testClient.getApiClient()
self.services = Services().services

@classmethod
def setUpClass(cls):
cls.api_client = super(TestPublicIP, cls).getClsTestClient().getApiClient()
cls.services = Services().services

29. Use assertions wherever possible for product failures, avoid using

30. Any new method added, make sure to have proper return codes wherever possible. We are using exceptions for treating the fail case , Using exceptions has a different purpose and is good to have exception handling, but please dont assume that raising exception to be a return value for a function or method for noting failure. Instead, provide explicit return values for new code added wherever possible. Check the return codes and then raise assertion or exception accordingly.