Why use a Custom Runner
Custom runners allow developers to extend Flex Unit 4 in order to run tests in a specific manner. One example of a custom runner is the parameterized runner. Instead of rewriting flex unit 4 to support parameterized testing, a custom runner was created to add support for running parameterized tests within the flex unit 4 framework. In other words, custom runners provide a hook for extensibility.
What is a Custom Runner
A custom runner] is a class that that implement IRunner
and is capable of running tests in a specific manner. Classes that implement the IRunner
interface must override the following methods:
*function run( notifier:IRunNotifier, previousToken:IAsyncTestToken ):void;
- Runs the test class and updates the notifier
on the status of running the tests.
-
- param
- notifier - The notifier that is notified about issues encountered during the execution of the test class.
- previousToken - The token that is to be notified when the runner has finished execution of the test class.
- throws
- org.flexunit.runner.notification.StoppedByUserException - The user has stopped the test run.
- param
*function get description():IDescription;
- Retruns an IDescription
of the test class that the runner is running.
In addition to implementing the abstract methods in IRunner
the custom runner must also provide a constructor that takes as an argument the Class
containing the tests.
Once a custom loader is built it must be linked in the test case. To link the custom loader into the test case, create a private variable of the custom loaders type. Also decorate the test case with a [RunWith]
metadata tag. For more information see Runners and Builders
Building a Custom Runner
Assumptions
''Note:'' This example assumes you have gone through basic test writing. If you have not, please read the Introduction.
About our Custom Runner
This custom runner is an advanced form of the FlexUnit .85 test runner that allows for before class and after class methods.
We will be designing our runner to observe the following patterns:
: 1) Accepts a method to be run before any tests in the class called "setupClass"
: 2) Accepts a method to be run before every test called "setup"
: 3) Accepts test methods which begin with "test"
: 4) Accepts a method to be run after every test called "tearDown"
: 5) Accepts a method to be run after all tests have completed called "tearDownClass"
Creating our custom runner class
The first step in creating our custom runner is to create a new custom runner class.
- In your test project, create a new package called 'runners'.
- Inside the runners package, create a new actionscript class called 'CustomRunner'. CustomRunner needs to implement the IRunner interface from FlexUnit 4.
Assuming you used FlashBuilder to create your class, your CustomRunner should look like this:If it does not you may copy this code into your runner. Theimport org.flexunit.runner.IDescription; import org.flexunit.runner.IRunner; import org.flexunit.runner.notification.IRunNotifier; import org.flexunit.token.IAsyncTestToken; public class CustomRunner implements IRunner { public function CustomRunner() { } public function run(notifier:IRunNotifier, previousToken:IAsyncTestToken):void { } public function get description():IDescription { } public function pleaseStop():void { } }
run
,get description
,pleaseStop
are part of theIRunner
interface and must be implemented by any custom runners. To add filtering and sorting options you may also implement theISortable
andIFilterable
interfaces. The implementation of these interfaces will not be covered in this tutorial.
Implementing the constructor
Obliviously, right now this runner will do nothing. The first thing that needs to be done is to save the test class information as a class
variable so can runner access it as needed. This is done by passing the class information to the runner. We also will need a copy of the class as a Klass
and an instance of the class on which to invoke methods for use later.
Modify the constructor as follows:
protected var clazz : Class; protected var clazzInfo : Klass; protected var testClass : Object; public function CustomRunner( clazz : Class ) { this.clazz = clazz; this.clazzInfo = new Klass( clazz ); this.testClass = new clazz(); }
What we've done:
- Modified the constructor to accept a
Class
. - Saved the
Class
that was passed in to the constructor as a protected variable. - Created an instance of a
Klass
based on theClass
passed to the constructor. - Saved the new
Klass
instance as a protected variable. - Created an instances of the
Class
passed in. - Saved the new instances of the
Class
as a protected variable.
Adding to the Constructor
Since we are creating a custom runner with setup and teardown methods for before and after each method and class, we will need to save a copy of those methods to call later. This is where the clazzInfo
will be useful. You will need to add a parser function called getSetupTearDown
. This call should be made as part of the constructor and should save the corresponding methods. Modify your code as follows:
protected var clazz : Class; protected var clazzInfo : Klass; protected var testClass : Object; protected var setupClass : Method; protected var setup : Method; protected var tearDownClass : Method; protected var tearDown : Method; public function CustomRunner( clazz : Class ) { this.clazz = clazz; this.clazzInfo = new Klass( clazz ); this.testClass = new clazz(); getSetupTearDown(); } protected function getSetupTearDown() : void { var methods:Array = clazzInfo.methods; var name:String; for ( var i:int=0; i<methods.length; i++ ) { name = ( methods[ i ] as Method ).name; if( name.substr( 0, 13 ) "tearDownClass" ) { tearDownClass = ( methods[ i ] as Method ); } else if( name.substr( 0, 10 ) "setupClass" ) { setupClass = ( methods[ i ] as Method ); } else if ( name.substr( 0, 5 ) "setup" ) { setup = ( methods[ i ] as Method ); } else if ( name.substr( 0, 8 ) "tearDown" ) { tearDown = ( methods[ i ] as Method ); } } }
:With this setup, the runner will look for any methods with the name "setupClass" and assume that this method should be called before any tests are run. The method named "tearDownClass" will be run after all tests in this class are complete. The methods "setup" and "tearDown" will be run before and after each test, respectively.
What we've done:
- Created a new function named
getSetupTearDown()
which- Iterates through all of the methods in the
clazzInfo
instance. - Saves the setup/teardown methods to their appropriate private variables.
- Iterates through all of the methods in the
- Made a call to
getSetupTearDown()
from the constructor.
Implement get description
Next, we will use the get description
method to get the IDescription
for the test class. This method will parse through the class and build a description for all tests contained in the suite. You will also need to create a helper method called describeChild
which will parse the IDescription
of each test method.
protected var cachedDescription : IDescription; protected var methodDescriptions : Array; protected var testMethods : Array; public function get description():IDescription { if( !cachedDescription ) { cachedDescription = Description.createSuiteDescription( clazz ); var methods : Array = clazzInfo.methods; testMethods = new Array(); for( var i : int = 0; i < methods.length; i++ ) { var method : Method = methods[ i ] as Method; if( method.name.substr( 0, 4 ) "test" ) { cachedDescription.addChild( describeChild( method ) ); testMethods.push( method ); } } methodDescriptions = cachedDescription.children; } return cachedDescription; } protected function describeChild( method : Method ) : IDescription { return Description.createTestDescription( clazz, method.name ); }
:You will notice we have created 3 new variables: cachedDescription
, methodDescriptions
, testMethods
. We will save our test class description in the cachedDescription
to save cycles later. The methodDescriptions
array contains the description for each individual test method. The testMethods
array contains references to each individual test method. This information will be used by the runner to run each test. By keeping a class reference to them, we save valuable cycles later.
What we've done:
*Created a method called describeChild
which parses the IDescription
of a given method.
*Implemented get description
function so it creates a description of the test. It does so by:
**Generating a description of the test class.
**Adding a description of each test method to the test class description.
Implement Run
So far we have created everything to set up our runner for running tests. However, if we were to use our custom runner at this point none of our tests would run. The IRunner
interface contains a method called run
which is invoked by the framework to actually run the test methods. We will need to set up our sequence here. See sequences for more information on sequence implementation.
public function run(notifier:IRunNotifier, previousToken:IAsyncTestToken):void { var method:Method; var methodDescription : IDescription; var name:String; //Run the setupClass method. if( setupClass ) setupClass.invoke( testClass ); //Run each test including setup and teardown. for( var i : int = 0; i < testMethods.length; i++ ) { method = testMethods[ i ] as Method; methodDescription = methodDescriptions[ i ] as IDescription; runChild( method, methodDescription, notifier ); } //Run the tearDownClasss if( tearDownClass ) tearDownClass.invoke( testClass ); //Notify the framework testing is complete, passing the token back. previousToken.sendResult(); } public function runChild( method : Method, description : IDescription, notifier : IRunNotifier ) : void { try { if( setup ) setup.invoke( testClass ); method.invoke( testClass ); } catch( error : Error ) { notifier.fireTestFailure( new Failure( description, error ) ); } finally { if( tearDown ) tearDown.invoke( testClass ); notifier.fireTestFinished( description ); } }
What we've done:
- Created the method
runChild()
which will:- Run the tests class's setup method if one exists.
- Run the given test method.
- Run the test class's teardown method if one exists.
- Notify the framework when a test fails.
- Implemented the
run
method which will:- Run the tests class's setupClass method if one exists.
- Iterate over each test method in the test class to:
- Notify the framework that the current test has begun.
- Call the
runChild()
method to execute the current test. - Notify the framework that the current test has ended.
- Run the tests class's tearDown method if one exists.
And there you have it. Put all of this code together and you have a custom runner which will may be used on any test class that follows our defined patterns. For information on actually using this custom runner, see [RunWith].