The testsuite should consist of two types of tests:
- Conept tests (testing general features of the plugin)
- Issue tests (Used to reproduce issues that were reported to the plugins Jira)
The framework for actually executing the tests should be TestNG.
The tests should be located in a package structure, that identifies them as concept or issue tests.
The tests rely on little maven projects that are configured to use the feature a concept test is using or to reproduce the bug an issue test should verify.
Inside the tests the Invoker Plugin is used to trigger a maven build using the current version of the maven-flex-plugin (http://maven.apache.org/shared/maven-invoker/)
This plugin is able to setup a test-local-repo and to have maven execute a build.
To make it easier to configure the creation of the local-repo, I would suggest using a so-called Mock Repository Manager as described here (http://mojo.codehaus.org/mrm-maven-plugin/examples/invoker-tests.html)
The TestNG test for an issue or a concept should contain a detailed description of the Concept being tested or the Issue that the test shoud reproduce. In Flexmojos the testsuite was missing this information and it was a nightmare to maintain the tests, if you don't really know what they should be doing.