Page History

Parallel Processing and Ordering

It is a common requirement to want to order message processing yet at the same time load balance processing in a SEDA style way across multiple threads, processes and machines.

How to achieve parallel processing

You can send messages to a number of Camel Components to achieve parallel processing and load balancing such as

• SEDA for in-JVM load balancing across a thread pool]
• ActiveMQ or JMS for distributed load balancing and parallel processing
• JPA for using the database as a poor mans message broker

Issues of ordering with parallel processing

As soon as you send multiple messages to different threads or processes you will end up with an unknown ordering across the entire message stream as each thread is going
to process messages concurrently.

...

A good rule of thumb to help reduce concurrency problems is to make sure each single can be processed as an atomic unit in parallel (either without concurrency issues or using say, database locking); or if it can't, use a Message Group to relate the messages together which need to be processed in order by a single thread.

Using Message Groups with Camel

To use a Message Group with Camel you just need to add a header to the output JMS message based on some kind of Correlation Identifier to correlate messages which should be processed in order by a single thread - so that things which don't correlate together can be processed concurrently.

...