In order to make the system more stable, we require committers and contributors to add automated test cases along with patches.  Reviewers should ensure that newly added code has sufficient test coverage, in a case-by-case base.

Currently, there are three kinds of automated tests in Apache AsterixDB:

  • Unit tests:  Unit tests are typically written in Java, using the JUnit Framework as well as the MockIto Framework. Here are a few conventions of unit tests:
    – Class name:  If the unit test is to test the class called "Foo", the name of the test class should be "FooTest".

    – Package and location: If the class under testing is in "asterixdb-bar/src/main/java/org/apache/asterixdb/bar/Foo.java", the test class should be placed in "asterixdb-bar/src/test/java/org/apache/asterixdb/bar/Foo.java".  Make sure the package names are the same, such that testing methods with "protected" and "default" access modifiers are easy.

    – Implementation: a unit test case typically is to only test a method in an isolated environment, where you can provide mocked inputs and fields using the MockIto framework. Typically, you can verify: (1) return values, (2) internal method call counts, and (3) raised exceptions.

    – Example unit tests:
       JobManagerTest
       SequentialFirstRuleCheckFixpointRuleControllerTest (using PowerMock to mock "final" methods.)
       TypeComputerTest
       ExceptionTest
       APIFrameworkTest (using PA.invoke(...) to call private methods)

    – Example patches (search "Test" in the patches)
       https://asterix-gerrit.ics.uci.edu/#/c/1432/
       https://asterix-gerrit.ics.uci.edu/#/c/1424/ 
       https://asterix-gerrit.ics.uci.edu/#/c/1377/

    – MockIto reference: http://site.mockito.org/

    – PowerMock reference (for mocking static, final, private etc.): https://github.com/powermock/powermock 

  • End-to-end tests:  End-to-end tests typically are queries or REST requests that runs against an AsterixDB instance. More specifically, end-to-end tests include:
    – RuntimeTest (asterixdb/asterix-app/src/test/resources/runtimets/): verifies that queries can return expected results or errors. 

    – OptimizerTest (asterixdb/asterix-app/src/test/resources/optimizerts/): verifies that the optimized query plans of queries are as expected.

    – MetadataTest (asterixdb/asterix-app/src/test/resources/metadata): verifies that metadata entities are as expected after executing DDL statements.

    – IntegrationTest : verifies that the installer, crash recovery, and queries can work as expected in a multi-JVM AsterixDB instance.

  • Performance test: Performance tests are run daily on AWS and the performance characteristics are shown at: http://asterix-perf.ics.uci.edu/.
    Performance tests are extensible, you can add more queries into existing benchmarks or add your new benchmarks at: asterixdb/asterix-benchmark/src/main/resources/benchmarks/, without knowing how instances are launched, how queries are submitted to the instance, and how execution times are reported.  
  • No labels