DRAFT

This page summarizes points to consider when adding or revisiting tests for some feature or functionality. Tests should be omitted only if there are certain reasons for not implementing tests of some kind.

1. Single threaded full API tests
   1. Single threaded API calls with proper assertions
   2. Single threaded async API calls with proper assertions
   3. Input parameters validation
   4. Configuration parameters validation
   5. Single node topology
   6. Multi nodes topology
   7. Operations from server nodes
   8. Operations from client nodes
   9. Cache modes
      1. partitioned, replicated, local
      2. atomic, transactional
      3. tx concurrency and isolation modes
   10. Cache memory modes
       1. onheap, 
       2. offheap
   11. Cache backups count - 0, 1, 2, ..., N
   12. Cache with store
   13. Cache with swap
   14. Cache with eviction policies
   15. Cache with exipry policies
   16. Marshallers - JDK, Binary, Optimized
   17. Peer class loading - On, Off
   18. Different key and value types - primitives, arrays, collections, POJOs, Serializables, Externalizables, Binarylizables.
2. Failover tests
   1. Multithreaded various API calls involving random branch choices with proper assertions
   2. Should be performed on changing topology
   3. May involve following technics
      1. Certain messages delaying
      2. Communication problems simulation
      3. Exception throwing from user implemented logic (e.g. SPIs, listeners, filters, interceptors etc).
   4. For combinations to cover see pt.1
   5. Tests should also be targeted to discover memory usage problems and possible memory leaks (in case of batching, deferred buffers, etc)
3. Benchmarks and performance tests
   1. Yardstick benchmark should be added
4. Long running performance tests
   1. May involve fault-tolerance test scenarios
   2. Should also address possible memory usage problems
   3. Should check that system parameters (throughput, latency, CPU, memory, IO, etc) are stable.

Full API coverage enhancements

There is a plane to have Full API test coverage of Ignite unctionality. See:  Unable to render Jira issues macro, execution error. .

The purpose of this ticket is to make sure that all ignite operations should work for any configuration properties combination.

Ignite already has Full API test suites, but currently it's hard to test all configuration combinations.