Why use a Custom Runner

Custom runners allow developers to extend Flex Unit 4 in order to run tests in a specific manner. One example of a custom runner is the parameterized runner. Instead of rewriting flex unit 4 to support parameterized testing, a custom runner was created to add support for running parameterized tests within the flex unit 4 framework. In other words, custom runners provide a hook for extensibility.

What is a Custom Runner

A custom runner] is a class that that implement IRunner and is capable of running tests in a specific manner. Classes that implement the IRunner interface must override the following methods:

*function run( notifier:IRunNotifier, previousToken:IAsyncTestToken ):void; - Runs the test class and updates the notifier on the status of running the tests.

    • param
      • notifier - The notifier that is notified about issues encountered during the execution of the test class.
      • previousToken - The token that is to be notified when the runner has finished execution of the test class.
    • throws
      • org.flexunit.runner.notification.StoppedByUserException - The user has stopped the test run.

*function get description():IDescription; - Retruns an IDescription of the test class that the runner is running.

In addition to implementing the abstract methods in IRunner the custom runner must also provide a constructor that takes as an argument the Class containing the tests.

Once a custom loader is built it must be linked in the test case. To link the custom loader into the test case, create a private variable of the custom loaders type. Also decorate the test case with a [RunWith] metadata tag. For more information see Runners and Builders

Building a Custom Runner

Assumptions

''Note:'' This example assumes you have gone through basic test writing. If you have not, please read the Introduction.

About our Custom Runner

This custom runner is an advanced form of the FlexUnit .85 test runner that allows for before class and after class methods.
We will be designing our runner to observe the following patterns:
: 1) Accepts a method to be run before any tests in the class called "setupClass"
: 2) Accepts a method to be run before every test called "setup"
: 3) Accepts test methods which begin with "test"
: 4) Accepts a method to be run after every test called "tearDown"
: 5) Accepts a method to be run after all tests have completed called "tearDownClass"

Creating our custom runner class

The first step in creating our custom runner is to create a new custom runner class.

  • In your test project, create a new package called 'runners'.
  • Inside the runners package, create a new actionscript class called 'CustomRunner'. CustomRunner needs to implement the IRunner interface from FlexUnit 4.
    Assuming you used FlashBuilder to create your class, your CustomRunner should look like this: 
    
     import org.flexunit.runner.IDescription;
     import org.flexunit.runner.IRunner;
     import org.flexunit.runner.notification.IRunNotifier;
     import org.flexunit.token.IAsyncTestToken;

     public class CustomRunner implements IRunner {
          public function CustomRunner() {
          }

          public function run(notifier:IRunNotifier, previousToken:IAsyncTestToken):void {
          }

          public function get description():IDescription {
          }

          public function pleaseStop():void {
          }
     }
    If it does not you may copy this code into your runner. The run, get description, pleaseStop are part of the IRunner interface and must be implemented by any custom runners. To add filtering and sorting options you may also implement the ISortable and IFilterable interfaces. The implementation of these interfaces will not be covered in this tutorial.

Implementing the constructor

Obliviously, right now this runner will do nothing. The first thing that needs to be done is to save the test class information as a class variable so can runner access it as needed. This is done by passing the class information to the runner. We also will need a copy of the class as a Klass and an instance of the class on which to invoke methods for use later.

Modify the constructor as follows:


     protected var clazz : Class;
     protected var clazzInfo : Klass;
     protected var testClass : Object;

     public function CustomRunner( clazz : Class ) {
          this.clazz = clazz;
          this.clazzInfo = new Klass( clazz );
          this.testClass = new clazz();
     }

What we've done:

  • Modified the constructor to accept a Class.
  • Saved the Class that was passed in to the constructor as a protected variable.
  • Created an instance of a Klass based on the Class passed to the constructor.
  • Saved the new Klass instance as a protected variable.
  • Created an instances of the Class passed in.
  • Saved the new instances of the Class as a protected variable.

Adding to the Constructor

Since we are creating a custom runner with setup and teardown methods for before and after each method and class, we will need to save a copy of those methods to call later. This is where the clazzInfo will be useful. You will need to add a parser function called getSetupTearDown. This call should be made as part of the constructor and should save the corresponding methods. Modify your code as follows:


     protected var clazz : Class;
     protected var clazzInfo : Klass;
     protected var testClass : Object;
     protected var setupClass : Method;
     protected var setup : Method;
     protected var tearDownClass : Method;
     protected var tearDown : Method;

     public function CustomRunner( clazz : Class ) {
          this.clazz = clazz;
          this.clazzInfo = new Klass( clazz );
          this.testClass = new clazz();
          getSetupTearDown();
     }

     protected function getSetupTearDown() : void {
          var methods:Array = clazzInfo.methods;
          var name:String;
            
          for ( var i:int=0; i<methods.length; i++ ) {
               name = ( methods[ i ] as Method ).name;
                
               if( name.substr( 0, 13 ) "tearDownClass" ) {
                    tearDownClass = ( methods[ i ] as Method );
               }
               else if( name.substr( 0, 10 ) "setupClass" ) {
                    setupClass = ( methods[ i ] as Method );
               }
               else if ( name.substr( 0, 5 ) "setup" ) {
                    setup = ( methods[ i ] as Method ); 
               }
               else if ( name.substr( 0, 8 ) "tearDown" ) {
                    tearDown = ( methods[ i ] as Method ); 
               }
          }         
     }

:With this setup, the runner will look for any methods with the name "setupClass" and assume that this method should be called before any tests are run. The method named "tearDownClass" will be run after all tests in this class are complete. The methods "setup" and "tearDown" will be run before and after each test, respectively.

What we've done:

  • Created a new function named getSetupTearDown() which
    • Iterates through all of the methods in the clazzInfo instance.
    • Saves the setup/teardown methods to their appropriate private variables.
  • Made a call to getSetupTearDown() from the constructor.

Implement get description

Next, we will use the get description method to get the IDescription for the test class. This method will parse through the class and build a description for all tests contained in the suite. You will also need to create a helper method called describeChild which will parse the IDescription of each test method.


     protected var cachedDescription : IDescription;
     protected var methodDescriptions : Array;
     protected var testMethods : Array;

     public function get description():IDescription {
          if( !cachedDescription ) {
               cachedDescription = Description.createSuiteDescription( clazz );
                
               var methods : Array = clazzInfo.methods;
               testMethods = new Array();
                
               for( var i : int = 0; i < methods.length; i++ ) {
                    var method : Method = methods[ i ] as Method;
                    
                    if( method.name.substr( 0, 4 ) "test" ) {
                         cachedDescription.addChild( describeChild( method ) );
                         testMethods.push( method );
                    }
               }
                
               methodDescriptions = cachedDescription.children;
          }
            
          return cachedDescription;
     }

     protected function describeChild( method : Method ) : IDescription {
          return Description.createTestDescription( clazz, method.name );
     }

:You will notice we have created 3 new variables: cachedDescription, methodDescriptions, testMethods. We will save our test class description in the cachedDescription to save cycles later. The methodDescriptions array contains the description for each individual test method. The testMethods array contains references to each individual test method. This information will be used by the runner to run each test. By keeping a class reference to them, we save valuable cycles later.

What we've done:
*Created a method called describeChild which parses the IDescription of a given method.
*Implemented get description function so it creates a description of the test. It does so by:
**Generating a description of the test class.
**Adding a description of each test method to the test class description.

Implement Run

So far we have created everything to set up our runner for running tests. However, if we were to use our custom runner at this point none of our tests would run. The IRunner interface contains a method called run which is invoked by the framework to actually run the test methods. We will need to set up our sequence here. See sequences for more information on sequence implementation.

     public function run(notifier:IRunNotifier, previousToken:IAsyncTestToken):void {
          var method:Method;
          var methodDescription : IDescription;
          var name:String;
            
          //Run the setupClass method.
          if( setupClass )
               setupClass.invoke( testClass );
    
          //Run each test including setup and teardown.     
          for( var i : int = 0; i < testMethods.length; i++ ) {
               method = testMethods[ i ] as Method;
               methodDescription = methodDescriptions[ i ] as IDescription;
                
               runChild( method, methodDescription, notifier );
          }
    
          //Run the tearDownClasss      
          if( tearDownClass )
               tearDownClass.invoke( testClass );
        
          //Notify the framework testing is complete, passing the token back.   
          previousToken.sendResult();
     }

     public function runChild( method : Method, description : IDescription, notifier : IRunNotifier ) : void {
          try {
               if( setup )
                    setup.invoke( testClass );
               method.invoke( testClass );
          }
          catch( error : Error ) {
               notifier.fireTestFailure( new Failure( description, error ) );
          }
          finally {
               if( tearDown )
                    tearDown.invoke( testClass );
               notifier.fireTestFinished( description );
          }
     }

What we've done:

  • Created the method runChild() which will:
    • Run the tests class's setup method if one exists.
    • Run the given test method.
    • Run the test class's teardown method if one exists.
    • Notify the framework when a test fails.
  • Implemented the run method which will:
    • Run the tests class's setupClass method if one exists.
    • Iterate over each test method in the test class to:
      • Notify the framework that the current test has begun.
      • Call the runChild() method to execute the current test.
      • Notify the framework that the current test has ended.
    • Run the tests class's tearDown method if one exists.

And there you have it. Put all of this code together and you have a custom runner which will may be used on any test class that follows our defined patterns. For information on actually using this custom runner, see [RunWith].

  • No labels