Versions Compared

Old Version 2

changes.mady.by.user Santhosh Kumar Edukulla

Saved on

compared with

New Version Current

changes.mady.by.user Santhosh Kumar Edukulla

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Marvin as a framework provides few facilities viz., deploying a datacenter, deleting a datacenter, running tests(both selfservice and provisioning), generating api information using spec file, retrieve and generate api information through cs endpoint  etc. Using nose, python unit testing module, and few other python packages, marvin framework provides these facilitation. To use these facilities, users need to be aware of few usages of marvin, few python packages, basic installation etc.

To simplify usage of these facilities, marvincli will be added. using this new cli, all the above tasks mentioned can be easily achieved. It provides an easy to use command and cli approach for users. marvincli, gets installed as part of marvin installation, so no new additional installation is required and marvin still continues to function as it is in its current form.

So, once marvin is installed, users can run the following commands using the new cli:

S.NoCommand NameCommand textDescription 
1.deploy

marvincli deploy config-file=<marvin-config-file>

EX: marvincli deploy config-file=setup/dev/advanced.cfg

Deploys a data center under CS.

Args:

  1. *config-file:config file to use for deploying the datacenter. EX: setup/dev/advanced.cfg
 
2.generatepis<fromspecfile>

marvincli generateapis api-spec-file= <api-spec-file> folder-path=<path where the output cloudstackAPI information is dumped>

EX: marvincli generateapis api-spec-file=/etc/cloud/cli/commands.xml folder-path=tools/marvin/marvin/

Generates the CloudStack API directory containing CS api information references, which is further used by marvin when making api calls to CS. It uses /etc/cloud/cli/commands.xml to generate these.

Args:

  1. api-spec-file: sample spec file path as similar to one /etc/cloud/cli/commands.xml. If no spec-file is provided, it uses /etc/cloud/cli/commands.xml file.
  2. folder-path : output folder path where cloudstackAPI directory will be created with all cs api reference information. If no path is provided, it will use the default path as tools/marvin/marvin to generate these.

Note: This currently, is generated as part of mvn profile, with this we can remove the particular mvn profile.

 
3.generatepis<fromendpoint>

marvincli generateapis end-point=<CS Endpoint ip> folder-path=<path where the output cloudstackAPI information is dumped>

EX: marvincli generateapis end-point=localhost folder-path=tools/marvin/marvin/


Generates the CloudStack API directory containing CS api information references, which is further used by marvin when making api calls to CS. Host ip of CS machine, is used to retrieve this information.

Args:

  1. *end-point: ip or name of the cs end point to be used to retrieve api information
  2. folder-path : output folder path where cloudstackAPI directory will be created with all cs api reference information. If no path is provided, it will use the default path as tools/marvin/marvin to generate these.

Note: This currently, is generated as part of mvn profile, with this we can remove the particular mvn profile.

 
 4.deploydc_and_runtest

marvincli deploydc_and_runtest config-file=<path_to_marvin_cfg> tc-path=<test suite or test case folder path> zone=<name of the zone> hyp-type=<hypervisor_type> required_hardware=<true\false>

EX: marvincli deploydc_and_runtest config-file=setup/dev/advanced.cfg tc-path=test/integration/smoke zone=sandbox-siumulator hyp-type=simulator required-hardware=false

Deploys a datacenter "and" runs test cases as well.

Args:

  1. *config-file: Config file path to deploy dc and for running tests. EX: setup/dev/advanced.cfg. Its a mandatory argument. If not provided, it will throw error and exit.
  2. *tc-path: Test Case folder or file path to pick tests from.
  3. zone: Name of the zone to be used for running test cases. If nothing is provided, picks up the first zone inside cfg for testing.
  4. hyp-type: Signifies hypervisor type to be used for running tests, EX: xenserver,kvm,vmware etc. So, if we have four different hosts under different clusters, this will help to run tests against a particular hypervisor type, helps in running multiple tracks against multiple hypervisors at a given time. If nothing is provided, will pick up the first hypervisor from config file.
  5. required-hardware: If set to False, picks up test cases which does not require hardware to run, helps in running when we dont have simulator and to test business logic of cs.
 
5.--cli-help

marvincli --h

(or)

marvincli --cli-help

Prints the marvincli help. Its commands usage and args information in colored text.

 
6.--version

marvincli --v

(or)

marvincli --version

 

Prints the marvincli version information, as similar to marvin version information. 
7.build_and_install

marvincli build_and_install cs-folder-path=<cloudstack code root dir > api-spec-file=<api spec file generated by CS EX: /etc/cloud/cli/commands.xml>

EX: marvincli build_and_install cs-folder-path=/root/cs-4.5/cloudstack/ api-spec-file=EX: /etc/cloud/cli/commands.xml

Builds and install marvin. It uses api spec file

EX: /etc/cloud/cli/commands.xml

file to build cs apis, used for marvin reference and then installs marvin.

Args:

*1. cs-folder-path : CS code root folder path EX: /root/cs-4.5/cloudstack/

2. api-spec-file: api spec file generated by CS EX: /etc/cloud/cli/commands.xml

Note : api-spec-file is not a mandatory field. If nothing is provided, it will use /etc/cloud/cli/commands.xml

 
8.sync_and_install

marvincli sync_and_install cs-folder-path=<cloudstack code root dir > end-point=<CS installed host ip>

EX: marvincli sync_and_install cs-folder-path=/root/cs-4.5/cloudstack/ end-point=localhost

 

Builds and install marvin. Its the same as previous command except that it uses an installed CS endpoint to generate apis, build and install marvin instead.

EX: /etc/cloud/cli/commands.xml

file to build cs apis, used for marvin reference and then installs marvin.

Args:

*1. cs-folder-path : CS code root folder path EX: /root/cs-4.5/cloudstack/

2. end-point: CS installed host ip EX: localhost

Note : end-point is not a mandatory field. If nothing is provided, it will use localhost

 
9.runtest

marvincli runtest config-file=<path_to_marvin_config> tc-path=<path to test case suite or test suite directory> zone=<name of zone> hyp-type=<xenserver or kvm or vmware> required_hardware=<true\false>


EX: marvincli runtest config-file=setup/dev/advanced.cfg tc-ath=test/integration/smoke zone=sandbox-siumulator hyp-type=simulator required-hardware=false

Runs a single test suite or list of suites, mentioned either smoke or component mentioned with below options.

Args:

  1. *config-file: Config file path to deploy dc and for running tests. EX: setup/dev/advanced.cfg. Its a mandatory argument. If not provided, it will throw error and exit.
  2. *tc-path: Test Case folder or file path to pick tests from.
  3. zone: Name of the zone to be used for running test cases. If nothing is provided, picks up the first zone inside cfg for testing.
  4. hyp-type: Signifies hypervisor type to be used for running tests, EX: xenserver,kvm,vmware etc. So, if we have four different hosts under different clusters, this will help to run tests against a particular hypervisor type, helps in running multiple tracks against multiple hypervisors at a given time. If nothing is provided, will pick up the first hypervisor from config file.
  5. required-hardware: If set to False, picks up test cases which does not require hardware to run, helps in running when we dont have simulator and to test business logic of cs.
 

Sample Workflow:

  1. Build CloudStack.             EX: mvn -P developer clean install
  2. Build and Install Marvin.   EX: marvincli cs-folder-path=/root/cs-4.5/cloudstack/ 
  3. Deploys a DataCenter.    EX:  marvincli deploy config-file=setup/dev/advanced.cfg
  4. Runs Test Cases.            EX:  marvincli runtest config-file=setup/dev/advanced.cfg zone=sandbox-siumulator hyp-type=simulator tc-path=test/integration/smoke required-hardware=false 

This will help take few marvin facilities easily from cli, printing its help and using it from cli iteself, with out looking at wikis and other information required.

Currently, there are few mvn profiles under tools/marvin/pom.xml, viz., marvin.sync, marvin.setup,marvin.test etc which are dealing with marvin building,usage etc. These usages are available through mvn commands through its profiles. We can now remove these profiles and decouple these usages of marvin from mvn, instead use the above cli commands for ease and maintenance. 

Below is the outline of few items we did last week along with brief plan going ahead.

1. Bug Fixes\Enhancements:
We did few bug fixes to Marvin. Debugged, analyzed and fixed bug fixes to Marvin.
Did code clean up in various areas under Marvin.
Enhancements as per request to Marvin. Enhanced few libraries\utilities for easy testing.
Added bugs to Jira for bug fixes.
Few patch submissions for enhancements
Did few nosetests plugin changes.
Marvin Refactor enhancements, testing the changes. ETA: Should be finished by end of this week.
Added enhancements to documentation, fixing documentation issues.
Educating the QA\Team for cleaner reviews.
We raised few bugs against product.

2. Reviews:
Tracking weekly review list.
Working with reviewers to get the reviewers issues fixed, Marvin changes required.
Did group reviews, reviewed feature tests, submitted issues, raised few patch submissions.
Closed and Submitted few review patches.

3. Misc:
Helping team with Marvin issues raised in ML,or otherwise to QA.
Fixed few issues reported.
Meetings and Misc tasks.
Jenkins: Setting up,Identifying of issues related to jenkins runs, fixing few flows etc

4. Working Plan:
Raising and fixing issues for Marvin..
Enhance logging support to Marvin. Current logging is not clean enough.
Helping team with new infrastructure related changes.
Adding config support to Marvin. Currently, there is no way to get clean configuration driven tests. We are planning to add this support to Marvin.
Support parallel deployment for new infrastructure support, logged a bug to track this, this request was raised from team for new infrastructure support. Requires some good effort on this change and testing.
Cleaning up of code, there were few areas where we are cleaning up unwanted code wherever is necessary and modularizing the structure.
Getting clean reviews and zero script errors. ( This is one of the goal, ultimately script errors should be zero )
Removing hard coding of values and separation from code. Currently, there is an issue raised from QA team and working to fix these as well.
Simulator changes. We already raised our availability to Alex\Sudha for further discussions. Plan is to start meeting on those to kick start to put up the plan and start implementing those changes.
Plan to study,work and get code coverage for CS using integration tests, we need to spend some time on getting code coverage tools and analysis using python.
Help in enhancing unit test code coverage wherever possible for CS. Enhancements to unit testing is a long term goal to get more coverage and release less bugs to QA.

Note: The list of bugs raised, fixed and patch submissions done should be available under jira\review board under cloudstack. We can provide them here if required.