Camel 3.0 roadmap
This is a roadmap which details the overall and major goals for Camel 3.0. Fell free to discuss this at the Camel Mailing Lists if you have ideas or feedback.
The Camel PMC conducted a survey in Oct 2010 to get a better understanding of how Camel is used and what the priorities for Camel 3.0 should be.
Nothing is committed/confirmed
The items listed on this wiki page is just ideas for the roadmap. Anything is subject for change.
Speak up if you want to ensure a feature is on the roadmap, or if you have new ideas etc.
When development of Camel 3.0 starts we will update status here which items have been implemented in the source code, and which have been discarded/deferred etc.
Clearer Architecture of Camel Core
Goals:
- The camel components should know as little as possible about camel core
- The classes needed to setup camel should be separate from the things needed at run time
So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
Split camel-core into three parts:
api, builder, impl
These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
Allowed depdencies ( "->" means may use, may depend on):
- * -> api
- end user config code -> builder
- builder -> impl
Define scope and rules for camel-core packages
In extension to the previous paragraph each camel package should have a clear scope that defines what to put in the package and what not. There should be rules that define what dependencies are allowed for classes in a package. The minimum goal is to guarantee that by following the rules dependency cycles can not happen. Additionally the rules should minimize dependencies between packages to achieve loose coupling between packages and high coherence inside a package.
Routing engine optimization
The internal routing engine should be optimized. See more details at Camel 2.x Speed optimizations.
More flexible routes at runtime
When routes is added in Camel 2.x architecture, global cross cutting concerns such as error handlers, interceptors, onCompletion etc. is applied when the route is added. We need to separate this and have those applied during routing. The Channel
needs to do this and therefore it must be more dynamic than its currently is. And we need to enlist the various global cross cutting concerns by their xxxDefintions in the CamelContext, so we can access them at any time. This allows end users also much more easily to add/remove interceptors, error handlers and whatnot at runtime. And it makes it much easier to add routes generated from JAXB or other sources, as we don't need to prepare or anyhow mold the RouteDefinition
given. See ticket CAMEL-3024 for some details.
Route initialization logic for Java DSL and XML DSLs
The Java DSL does its route initialization slightly a bit different than the XML DSLs, due the nature of it, and the fact the fluent builders can do additional logic, which the JAXB model of XML DSLs does not. We should align the initialization logic so Java DSL and XML DSLs does the same thing. They setup the pure model at first. So the configure method in the RouteBuilder should setup the model as the XML DSL would do. Then the prepare route logic which follows could be the same in all cases. This would also allow us to ensure when people use multiple RouteBuilder classes in Java DSL, then context scoped onException, interceptors is applied for all RouteBuilders.
Add OnException, Interceptor, etc. to JAXB model for a CamelContextDefinition
Configuring context scoped onException, interceptors etc. is woven into the RouteDefinition as part of the route initialization logic. When we have a dynamic routing engine (see above) that can at runtime support this without the need for woven into the routes. Then we should also ensure the context scoped onException, interceptors etc. is available in a CamelContextDefinition. This ensures the models is always 100% kept as it was provided, and we can fully export the model to XML and other languages (having a supported render).
Tighten up route definitions
Currently cross cutting concerns such as error handlers, interceptors, onCompletion etc. can be define anywhere in the route. We should tighten this up and only allow this to be configured in the start of the route. This also ensures when end users use code assistance in their route development, the IDE will not popup a big list which includes these cross cutting concerns. See also next note. (ProcessorDefinition will therefore be trimmed)
Support for asynchronous transactions
When using the asynchronous routing engine it would be desirable of transactional context could be propagated to the new threads.
This requires the TX manager supports suspend/resume on the TX. G.Nodet have worked a bit on this. See CAMEL-2902. Also see CAMEL-2729.
With the Asynchronous Routing Engine it would be great if we could support asynchronous transaction as well. See CAMEL-2729 and CAMEL-2902
Remove @deprecated
@deprecated features, methods, etc. is to be removed.
Stream caching
We could add support for using HawtDB as the persistent store for streams which overflow to disk store.
EIP
The Resequencer EIP currently doesn't support persistence, we could introduce this and let it leverage HawtDB such as we did with the Aggregator2 EIP.
Schedule in DSL
We could consider adding DSL syntax sugar for scheduling routes. For example currently you have to use Quartz or a ScheduledPollingConsumer
which has the delay
option. We could add DSL which has something like:
schedule().every(5).minute().pollFrom("xxx").to("yyyy")
The hard part is to come up with a good DSL syntax. We can look at BAM and see what we got there as well.
The DSL should support both cron and non cron based, eg Quartz, Spring (spring 3 has cron) and regular JDK timers.
Fix routes with multiple inputs
The current implementation of routes with multiple inputs is to clone the route, which means you essentially got 2+ routes if a route has multiple inputs. However routes with multiple inputs is seldom used. The correct implementation is to only create one route but have multiple input consumers. This change will require a bit of change in current code as it relies on the only 1 input consumer on the route.
Up-to-date Scala DSL
Done in Camel 2.11
The Scala DSL is slightly out of date as we have improved the DSL a bit here and there. We should check the gap and ensure the Scala is up-to-date.
Advanced Scala support
A proposal for advanced Scala support is scalaz-camel which is an alternative to Camel's existing Scala DSL.
More EIPs as @annotations
Currently its only the Routing Slip, Recipient List and Dynamic Router which are avail as @annotation as well. We could add more EIPs as annotations such as Splitter.
And also maybe annotations for AggregationStrategy
to make this less Camel API dependent, so you can use a plain POJO for that.
Unified statistics
Currently the performance statistics is only avail when using JMX. We should allow those stats to be enabled regardless if JMX is enabled or not. Then we can use those stats from the web console. This also allows to expose those stats in the cloud where JMX is often not possible to be used.
In the camel-jpa
component we could offer JPA annotated entities with the performance status (just as we do for the tracer). Then end user can more easily use that if they want the stats to be persisted in a database using JPA.
SEDA/VM components to leverage async routing engine
This allows to use non blocking request-reply over SEDA and VM. The reason why we havent converted in 2.4 is it causes a bigger API breakage.
camel-osgi-test
When testing your Camel apps with OSGi you may use PaxExam for that. We should create a test kit for osgi, like we have camel-test for regular junit testing. The test kit should make it easy for end users to have their apps tested with OSGi. We already have pieces in the tests/camel-itest-osgi
. We just need to clean and shape it up so its ready for end users as well. And of course add documentation as well.
And then we should use it in camel-itest-osgi
of course. See CAMEL-3777.
REST
We already have REST support with CXFRS and Restlet but it can be better. We should make sure those components is dead easy to use and you can invoke REST services in one line of code etc. And we should make more examples and tidy up the CXFRS documentation.
More load tests
More load tests for frequently used Camel components (jetty, jms ...) and camel-core.
- Ensure correct behaviour under load
- Source for performance numbers (throughput etc).
- Detection of memory leaks
- Detection of performance decreases after refactorings
- ...
Faster unit tests
Decreasing the total time for our complete build from more than 2 hours at present. This means in fact we have to decrease the time of our unit tests. At present, we have many unit test which starts Camel contexts and Camel routes to test data formats, components, endpoints, consumer ... This is not really necessary in most (or all) the cases.
OSGi enhancements
- create a single type converter registry available as OSGi service to all bundles serving as a single registry to lookup converters or to add/remove converters from custom bundles
Easier commit/rollback for component developers
Maybe expose some interface having commit / rollback methods to make it easier for component developers to implement custom logic. Currently they may not know about OnCompletion and how to use UnitOfWork
on Exchange to do this today.
Unify uri/ref
Instead of having both uri and ref for endpoints, we should unify this and only use uri. If people want to use ref, then they can do that using "ref:xx" as an uri. This would simplify code as we dont have to check for either one.
Less Spring dependencies
DONE in Camel 2.9
In camel-core we use the Spring JMX annotations to more easily enlist our MBeans. We should move that logic to camel-spring. And introduce a Camel specific annotations to replace those. For example ActiveMQ does that. This allows us to use camel-core with JMX without any spring JARs at all. End users can still use the Spring JMX annotations in their custom code / components. They just need camel-spring on the classpath.
Likewise we should move the ResourceEndpoint from camel-spring to camel-core. This ensures that resource loading on classpath works with Camel as we can leverage the ClassResolver. For example OSGi blueprint does not work with the Spring ResourceEndpoint. Also this ensures the components that uses ResourceEndpoint will no longer be dependent on Spring.
Then we are down to have Spring JAR dependency in: camel-jms and camel-mail.
Introduce Camel JMX annotations
DONE in Camel 2.9
See above about less Spring dependency.
Refactor UnitOfWork
The implementation of DefaultUnitOfWork seems to have transformed itself into a to broad concern where unit of work is doing a bit more work than the transactional aspect that ties to its name.
Maybe this implementation should be named ExchangeContext and we can introduce a simpler UnitOfWork concept. This would also allow us to refactor the SubUnitOfWork into a general parent/child unit of work concept.
However this requires API changes and thus is best kept for Camel 3.0
This is also needed by the refactor of the Message History EIP.
Improvements to ThreadPoolProfile for thread management
We could move ThreadPoolProfile
from org.apache.camel.spi
to org.apache.camel
and have it in the root package.
Tighten up onException
We should consider tighten up the onException DSL a bit, for example to make it more clear that if you have processing steps in there, they only occur after the message has been exhausted. So maybe we need to have a onExhausted to make this clear
onException(IOException.class).maximumRedeliveries(3) .onExhausted().handled(true).to("log:ignoreMe");
So in this example its only after the 3 failed redeliveries, then its exhausted, and then we handle the message and route it to a log where we ignore the message.
Also currently you can do this:
onException(IOException.class).maximumRedeliveries(3) .handled(true);
Which will handle the message and after this let the error handler deal with the message. See CAMEL-5059.
We should in this case use a NoopProcessor so the message is handled, but the regular error handler does not react. Then its consistent.
However we have not changed this in Camel 2.x to keep backwards compatibility.
Likewise there has been ideas to move onRedeliverRef to <redeliveryPolicy> as currently its to be configured outside the policy.
It may make more sense to move onRedeliverRef to the policy to keep it together.
Add composite EIP to compose a number of EIPs into a single EIP
It would be nice if we have a DSL to compose a number of child EIPs into a single EIP itself. eg a bit like <pipeline>. This would allow people to make it easier to group together a number of EIPs into a single "unit". This allows for example the error handler to redeliver to the composite EIP instead of at the point of failure within the group. There are use-cases where people want to do that. And today the solution is to split this into a new route, which you then disable error handler by setting it to no error handler. And then call the route using the direct endpoint. So instead if you could do <composite> ... stuff goes here </composite> then that would be neater, as you dont have to split into multiple routes. I think there is an old JIRA ticket created about this a long time ago.
Message History EIP
We should make this EIP easier to use for end users, but offering a better public API. And also have a pluggable message store, which filters that can filter what should be store. As well pluggable marshallers so people can marshal data from Exchange into a format the message store can store (BLOB, XML, JSon etc.).