Page History

Current State of Testing

Consensus on level of testing coverage at unit, integration, test level and objectives for what we should achieve

• Nayan: integration tests do cover most business scenarios but unit tests don't cover.
  • This isn't scalable - heavy dependency on developers to have to write this. 
  • Domain experts can write far better test cases - need framework for semi-technical individuals to write test cases. 
• Aleks: a lot of manualy boilerplate code throughout application
  • Manual classes in REST API - impedes effort of getting tests out there. 
  • Aleks: useful to use cucumber - 
• Disconnect between tickets, documentation and what is actually tested?
  • Difficult to understand test coverage
  • Only thing aleks can do is look at code and try to cross-read
  • With Cucumber cards can do annotation and any tags.
    • Can introduce standardized tags according to tags/modules
    • Tags could point to JIRA ticket and can create some stats
    • Also a good feature that converts reports into ASCII doc which will immediately. 
  • Why our integration tests are peculiar
    • Testing business logic at rest API level
    • Better to do at unit test so isolate and not have to ramp up entire Fineract
    • Task: make integration tests faster and should also migrate logic that is there to proper unit tests
      • Can then run these during development and not have to wait like 5/6 minutes which is time it takes to run test cases currently
        • Should be as a button click in dev environment - simply click and run. 

• Istvan: what overall do we want to improve?
• Aleks: Recently got very frustrated by build process and tests = long time and too complex to understand 
  • • Raw numbers for time to wait - compile - gradle kicks in always - no usage of gradle cache whatsoever - effectively wait between 5 and 6 minutes until tests are ready
    • Even just selecting one integration test case, takes 10 minutes. Might only have one or two test functions but a lot of code - tests are easily 1000 lines of code.
      • Not easy to grok idea of what's being tested. 
      • Overall an hour. to run in a consistent build
        • Added test containers but then took 2 hours without the optimizations
        • Ripped up build scripts - lot of manual task dependencies in place and removed all of that - some build steps might not be in right order but good 
        • Dependencies on quality checkers but do you need these all the time. 
        • Got build process down to under 20s with proper docker image. 
          • Previously with docker compose file - docker image takes 8 to 10 minutes to finish - no caching used - replaced with google gradle plugin Jit? 
          • Don't need tomcat plugin or additional mysql - can test easily with proper docker image. 
        • Sneaky configurations with java parameters that was tricky to figure out. 
          • Steps in build script to ensure not ignoring warnings/errors - settings were quite strict - array with parameters - build script different and then add more parameters - by splitting the paramters into two arrays - created a lot of waiting time. 
          • Task: create pull request for improving build script. 
          • Overall time - under 9 minutes for complete suite of tests and under 20s for build. 
    • 437 test cases but can't categorize where these are
      • Some areas where more things are tested.
      • INtegration tests aren't too bad but just very slow. 

Adam: Also in agreement with using Cucumber as well - other projects using it in-house for testing. 

• Previous project only wanted to see coverage through unit tests and not via integration tests. 
  • Ensure that unit tests were there for business logic as well - both positive and negative scenarios. 
• Adam: take a hybrid path - unit tests for critical business logic - charge calculation, prepayment calculation, cover the logic by unit tests
  • Integration tests - most beneficial way is to use is for regression and to check that all is intact and as designed.
  • Quality of the tests was very important to them - usually not happy with coverage initially so were focused on if it was really tested. Regression costs are extremely high so want to avoid at all costs and catch as early as possible.