Page History

Summary

We are proposing to develop a modular and performant command processing component in parallel to the currently available one in Fineract and gradually migrate REST API controllers/resource classes, command handlers and where necessary business logic services. These changes will improve the developer experience (speed) significantly by (re-)introducing proper Class types where we currently use Strings as a common denominator to process various command payloads. Having proper type information is a requirement to delegate all JSON related de-/serialization to the Jackson framework and ultimately get rid of all handcrafted JSON parsing boilerplate  code (ca. 20% of the overall code base). With that we'll regain error checking at compile time (vs. tedious bug hunting at runtime, because of e.g. a simple refactoring of a command attribute). We are expecting visible performance gains by using Jackson and having a simpler command processing pipeline that has configurable performance levels (sync, async, non-blocking).

Status  

Under Development

Who is Involved   

• Aleksandar Vidakovic (proposal owner, lead developer)
• Victor Romero (reviewer)
• Adam Saghy (reviewer)

Background/Motivation

Fineract accumulated some technical debt over the years. One area that is implicated is type-safety of internal and external facing APIs, the most prominent of which is Fineract's REST API. In general the package layout of the project reflects a more or less classic layered architecture (REST API, data transfer/value objects, business logic services, storage/repositories). The project predates some of the more modern frameworks and best practices that are available today and on occasions the data structures that are exchanged offer some challenges (e.g. generic types). Fineract's code base reflects that, especially where JSON de-/serialization is involved. Nowadays, this task would be simply delegated to the Jackson framework, but when Fineract (Mifos) started the decision was made to use Google's GSON library and create handcrafted helper classes to deal with JSON parsing. While this provided a lot of flexibility this approach had some downsides:

...

In conclusion, if we really want to fix those issues that are not only cosmetic and affect the performance and the developer experience equally then we urgently need to fix the way how we process write requests aka commands.

Goals

• new command processing will run independently next to the legacy mechanics
• self contained
• fully tested
• ensure that the REST API is 100% backward compatible
• try to contain the migration and make it as easy as possible for the community to integrate those changes
• introduce types where needed and migrate the (old) JAX-RS REST resource classes to Spring Web MVC (better performance and better test-ability)
• introduce DTOs if not already available and make sure if they exist that they are not outdated
• assemble one DTO as command payload from all incoming REST API parameters (headers, query/path parameters, request bodies)
• annotate attributes in the DTOs with Jakarta Validation annotations to enforce constraints on their values
• wired REST API to the new command processing, one service at a time/pull request
• take a non-critical service (like document management) and migrate it to the new command processing mechanics from top (REST API) to bottom (business logic service)
• refactor command handlers to new internal API
• make sure that the business service logic classes/functions take only one DTO request input parameter (aka don't let a function have 12 input parameters of type string...)
• when all integration tests run successfully then remove all legacy boilerplate code that is not used anymore
• make an ordered list of modules/features (easiest, lowest hanging fruit first)
• maintain at least the same performance as the current implementation
• optional: improve performance if it can be done in a reasonable time frame
• optional: improve resilience if it can be done in a reasonable time frame

Exclusions

• current command processing will stay untouched, will run independently of new infrastructure
• don't try cleaning up the storage layer; that's a separate effort for later (type safe queries, query performance, clean entity classes)
• doesn't need to be optimized for speed immediately
• no changes in the integration tests

Change Proposed

Command Wrapper

Class contains some generic attributes like:

...

The actual payload (aka command input parameters) are defined as a generic parameter. It is expected that the modules implement classes that introduce the payload types and inherit from the abstract command class.

Command Processing Service

• synchronously (required): this is pretty much as we do right now (use virtual threads optionally)
• asynchronously (optional): with executor service and futures (use virtual threads optionally)
• non-blocking (optional): high performance LMAX Disruptor non-blocking implementation

...

NOTE: we should consider providing a command processing implementation based on Apache Camel once this concept is approved and we migrated already a couple of services. They are specialized for exactly this kind of use cases and have more dedicated people working on it's implementation. Could give more flexibility without us needing to maintain code.

References to users

Keep things lightweight and only reference users by their user names.

Critical patch of execution

Again, keep things as lightweight as possible and avoid any database operations during the command processing.

Alternatives Considered

We could just opt to use an existing framework like Apache Camel which provides all necessary building parts to implement and efficient command processing infrastructure. We still recommend starting with a standalone solution that is properly abstracted with interfaces. Once the basic mechanics and the performance are tested we can still opt to provide an alternative implementation (aka command processing plugin) that is based on Apache Camel. That way we avoid any vendor (framework) lock-in.

Phases of work

• Jan 12-15 2025: First prototype and pull request available
• Jan 2025: community vote
• Jan 2025: pick an example Fineract service/feature (e.g. document management) that is less used than others and onboard devs
• Feb 2025: review the results, lessons learned, performance improvement (or degradation)
• Feb 2025: community vote if this is a viable approach; if yes, prioritize the migration of Fineract's services/features starting with the least important (Fineract has approximately 50 features/services/"modules" that would need to be migrated)
• Feb 2026: added audit capabilities
• Mar 2026: tentative date for completion of migration of 20 modules (currently ca. 10 modules are migrated or close to)

...