Test Guideline

In order to make the system more stable, we require committers and contributors to add tests along with patches.  Reviewers should ensure that newly added code has sufficient test coverage, in a case-by-case base.

Currently, there are three kinds of tests in Apache AsterixDB:

• Unit tests:  Unit tests are typically written in Java, using the JUnit Framework as well as the MockIto Framework. Here are a few conventions of unit tests:
  – Class name:  If the unit test is to test the class called "Foo", the name of the test class should be "FooTest".
  
  – Package and location: If the class under testing is in "asterixdb-bar/src/main/java/org/apache/asterixdb/bar/Foo.java", the test class should be placed in "asterixdb-bar/src/test/java/org/apache/asterixdb/bar/Foo.java".  Make sure the package names are the same, such that testing methods with "protected" and "default" access modifiers are easy.
  
  – Implementation: a unit test case typically is to only test a method in an isolated environment, where you can provide mocked inputs and fields using the MockIto framework. Typically, you can verify: (1) return values, (2) internal method call counts, and (3) raised exceptions.
  
  – Example unit tests:
     JobManagerTest
     SequentialFirstRuleCheckFixpointRuleControllerTest
     TypeComputerTest
     ExceptionTest
  
  – Example patches (search "Test" in the patches)
     https://asterix-gerrit.ics.uci.edu/#/c/1432/
     https://asterix-gerrit.ics.uci.edu/#/c/1424/ 
     https://asterix-gerrit.ics.uci.edu/#/c/1377/
  
  – MockIto reference: http://site.mockito.org/
  
  – PowerMock reference: https://github.com/powermock/powermock 
  
  
• End-to-end tests:  End-to-end tests typically are queries or REST requests that runs against an AsterixDB instance. More specifically, end-to-end tests include:
  – RuntimeTest (asterixdb/asterix-app/src/test/resources/runtimets/): verifies that queries can return expected results or errors. 
  
  – OptimizerTest (asterixdb/asterix-app/src/test/resources/optimizerts/): verifies that the optimized query plans of queries are as expected.
  
  – MetadataTest (asterixdb/asterix-app/src/test/resources/metadata): verifies that metadata entities are as expected after executing DDL statements.
  
  – IntegrationTest : verifies that the installer, crash recovery, and queries can work as expected in a multi-JVM machine AsterixDB instance.
  
  

• Performance test: Performance tests are run daily on AWS and the performance characteristics are shown at: http://asterix-perf.ics.uci.edu/.
  Performance tests are extensible, you can add more queries into existing benchmarks or add your new benchmarks at: asterixdb/asterix-benchmark/src/main/resources/benchmarks/, without knowing how instances are launched, how queries are submitted to the instance, and how execution times are reported.