1. Fixed few issues reported.
https://issues.apache.org/jira/browse/CLOUDSTACK-5963
https://issues.apache.org/jira/browse/CLOUDSTACK-5498
https://issues.apache.org/jira/browse/CLOUDSTACK-5674
https://issues.apache.org/jira/browse/CLOUDSTACK-5948
https://issues.apache.org/jira/browse/CLOUDSTACK-5973
https://issues.apache.org/jira/browse/CLOUDSTACK-6006
https://issues.apache.org/jira/browse/CLOUDSTACK-5980
https://issues.apache.org/jira/browse/CLOUDSTACK-6158
https://issues.apache.org/jira/browse/CLOUDSTACK-6160
https://issues.apache.org/jira/browse/CLOUDSTACK-6316
https://issues.apache.org/jira/browse/CLOUDSTACK-6257
https://issues.apache.org/jira/browse/CLOUDSTACK-6282
...
...
...
...
...
...
...
4. Logging enhancements: During the regression run,earlier we had various and a little confusing log options. Now, marvin run logs will be streamlined at one place. Here, for the entire run, we will see different logs viz., run\results\exceptions+passfail log all at one place. We have added MarvinLog to explicitly provide logging facility to Marvin. Before running test suites, we just need to configure the path for logs. MarvinInit along with MarvinLog will now create all logs required for the run under this path. Before every run, it will clean up the previously created MarvinLogs folder before proceeding with run. MarvinLogs folder will be created at the mentioned and configured path and all test suites will dump different logs here. Each test run will now have a test module name for log for easy identification. Logs for Run info, exception information ,results information etc are all provided here. So, all logs separated by different category help resolve issues quickly. The log folder path once configured, will be used to provide logs separately for each test module. All Test script Exceptions will now be caught at one place under exception\failed log. It will help identify script issues vs product issues. Added few more log levels here.
https://issues.apache.org/jira/browse/CLOUDSTACK-5498
...
...
...
Deploy DataCenter with multiple zones Say 3 zones one for each hypervisor( zoneforxen,zoneforkvm,zoneforvmware etc).
python deployDataCenter.py -i <input_cfg_file>
nosetests --with-xunit --xunit-file=xen_hypervisor_output.xml --with-marvin --marvin-config=/hudson/scripts/nightly_asf_master.cfg -w <path_to_smoke> -a tags=advanced --zone=<zoneforxen>
nosetests --with-xunit --xunit-file=xen_hypervisor_output.xml --with-marvin --marvin-config=/hudson/scripts/nightly_asf_master.cfg -w <path_to_smoke> -a tags=advanced --zone=<zoneforkvm>
nosetests --with-xunit --xunit-file=xen_hypervisor_output.xml --with-marvin --marvin-config=/hudson/scripts/nightly_asf_master.cfg -w <path_to_smoke> -a tags=advanced --zone=<zoneforvmware> etc.
3 . Added few new features and fixed few issues reported under marvin\test suites. More details under bugs below:
https://issues.apache.org/jira/browse/CLOUDSTACK-
...
https://issues.apache.org/jira/browse/CLOUDSTACK-5498
https://issues.apache.org/jira/browse/CLOUDSTACK-5674
https://issues.apache.org/jira/browse/CLOUDSTACK-5948
https://issues.apache.org/jira/browse/CLOUDSTACK-5973
https://issues.apache.org/jira/browse/CLOUDSTACK-6006
https://issues.apache.org/jira/browse/CLOUDSTACK-5980
https://issues.apache.org/jira/browse/CLOUDSTACK-6158
https://issues.apache.org/jira/browse/CLOUDSTACK-6160
https://issues.apache.org/jira/browse/CLOUDSTACK-6316
https://issues.apache.org/jira/browse/CLOUDSTACK-6257
https://issues.apache.org/jira/browse/CLOUDSTACK-6282
https://issues.apache.org/jira/browse/CLOUDSTACK-5674
4. Logging enhancements: All marvin related logs will be streamlined at one place for the total run. It will help in debugging failures post run. Now, for the entire run, and for each test suite, we will see different logs viz., "run\results\exceptions+passfail " log all at one place. So, all logs separated by different category help resolve issues quickly. identify product issues vs test failures. The log folder path once configured, will be used to provide logs separately for each test module.
6. Marvin Init Enhancements: Earlier, we didn't had Marvin initialization, the object of initialization is to create test client, parse configuration using config library, create logging structures, deploy DC, parse test data required. All these are done at initialization state and are made available to test suites through test client. Earlier test and data are little tied up through service class. Need to remove all data required from tests to be data driven. Based upon the config provided, tests will run now.
7. We added MarvinInit to initialize marvin earlier. Now, added few more changes to it. So, the init will exit based upon testclient creation, logging creation, config etc and exit.
8. cloudstackConnection.py: Did few changes to this module. Fixed a bug and there was some cluttered code earlier. Did some clean up here. marvinRequest used by cloudstackClientAPI for sending requests to CS will now return FAILED in case of any issue like exception, invalid arguments, etc. Test Modules can now verify whether the command is FAILED or not before proceeding further. Currently, at many places we just proceed with out verifying the output from marvinRequest.
9. ConfigManager: Changed few naming conventions, cleanup and config parameters.
10. Added few log levels to runlogger, so tests can dump the log type they wanted. Tests should use exception\error for dumping that level information compared to debug level.
11. Earlier, if test modules has issues with either invalid import, indentation etc were prepared to run by noseplugin. Added explicit check to import and see its importability and if not exit with log rather than proceeding further. Avoids unwanted exceptions from noseplugin for bad modules.
Note1: Now, related to note5, we need few changes to tests. I have logged and assigned bug to track this. Once this change is done, we will proceed to do more changes to run regression on new infrastructure.
Note2: Need to Test the changes, will soon move all these changes to topic branch created for marvin refactoring. Once "tested" and agreed , merge the changes with main branches of CS.
...
Phase2:
Planned to fix below issues issue\Enhancements for phase2.
https://issues.apache.org/jira/browse/CLOUDSTACK-6344
https://issues.apache.org/jira/browse/CLOUDSTACK-6383
https://issues.apache.org/jira/browse/CLOUDSTACK-6384
...