Apache Camel

User Guide

Version 2.15.0


Copyright 2007-2016, Apache Software Foundation

Introduction

Apache Camel ™ is a versatile open-source integration framework based on known Enterprise Integration Patterns.

Camel empowers you to define routing and mediation rules in a variety of domain-specific languages, including a Java-based Fluent API, Spring or Blueprint XML Configuration files, and a Scala DSL. This means you get smart completion of routing rules in your IDE, whether in a Java, Scala or XML editor.

Apache Camel uses URIs to work directly with any kind of Transport or messaging model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF, as well as pluggable Components and Data Format options. Apache Camel is a small library with minimal dependencies for easy embedding in any Java application. Apache Camel lets you work with the same API regardless which kind of Transport is used - so learn the API once and you can interact with all the Components provided out-of-box.

Apache Camel provides support for Bean Binding and seamless integration with popular frameworks such as CDISpring, Blueprint and Guice. Camel also has extensive support for unit testing your routes.

The following projects can leverage Apache Camel as a routing and mediation engine:

  • Apache ServiceMix - a popular distributed open source ESB and JBI container
  • Apache ActiveMQ - a mature, widely used open source message broker
  • Apache CXF - a smart web services suite (JAX-WS and JAX-RS)
  • Apache Karaf - a small OSGi based runtime in which applications can be deployed
  • Apache MINA - a high-performance NIO-driven networking framework

So don't get the hump - try Camel today! (smile)

Too many buzzwords - what exactly is Camel?

Okay, so the description above is technology focused.
There's a great discussion about Camel at Stack Overflow. We suggest you view the post, read the comments, and browse the suggested links for more details.

Quickstart

To start using Apache Camel quickly, you can read through some simple examples in this chapter. For readers who would like a more thorough introduction, please skip ahead to Chapter 3.

Walk through an Example Code

This mini-guide takes you through the source code of a simple example.

Camel can be configured either by using Spring or directly in Java - which this example does.

This example is available in the examples\camel-example-jms-file directory of the Camel distribution.

We start with creating a CamelContext - which is a container for Components, Routes etc:{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-jms-file/src/main/java/org/apache/camel/example/jmstofile/CamelJmsToFileExample.java}There is more than one way of adding a Component to the CamelContext. You can add components implicitly - when we set up the routing - as we do here for the FileComponent:{snippet:id=e3|lang=java|url=camel/trunk/examples/camel-example-jms-file/src/main/java/org/apache/camel/example/jmstofile/CamelJmsToFileExample.java}or explicitly - as we do here when we add the JMS Component:{snippet:id=e2|lang=java|url=camel/trunk/examples/camel-example-jms-file/src/main/java/org/apache/camel/example/jmstofile/CamelJmsToFileExample.java}The above works with any JMS provider. If we know we are using ActiveMQ we can use an even simpler form using the activeMQComponent() method while specifying the brokerURL used to connect to ActiveMQ

In normal use, an external system would be firing messages or events directly into Camel through one if its Components but we are going to use the ProducerTemplate which is a really easy way for testing your configuration:{snippet:id=e4|lang=java|url=camel/trunk/examples/camel-example-jms-file/src/main/java/org/apache/camel/example/jmstofile/CamelJmsToFileExample.java}Next you must start the camel context. If you are using Spring to configure the camel context this is automatically done for you; though if you are using a pure Java approach then you just need to call the start() method

camelContext.start();

This will start all of the configured routing rules.

So after starting the CamelContext, we can fire some objects into camel:{snippet:id=e5|lang=java|url=camel/trunk/examples/camel-example-jms-file/src/main/java/org/apache/camel/example/jmstofile/CamelJmsToFileExample.java}

What happens?

From the ProducerTemplate - we send objects (in this case text) into the CamelContext to the Component test-jms:queue:test.queue. These text objects will be converted automatically into JMS Messages and posted to a JMS Queue named test.queue. When we set up the Route, we configured the FileComponent to listen off the test.queue.

The File FileComponent will take messages off the Queue, and save them to a directory named test. Every message will be saved in a file that corresponds to its destination and message id.

Finally, we configured our own listener in the Route - to take notifications from the FileComponent and print them out as text.

That's it!

If you have the time then use 5 more minutes to Walk through another example that demonstrates the Spring DSL (XML based) routing.

Walk through another example

Introduction

Continuing the walk from our first example, we take a closer look at the routing and explain a few pointers - so you won't walk into a bear trap, but can enjoy an after-hours walk to the local pub for a large beer (wink)

First we take a moment to look at the Enterprise Integration Patterns - the base pattern catalog for integration scenarios. In particular we focus on Pipes and filters - a central pattern. This is used to route messages through a sequence of processing steps, each performing a specific function - much like the Java Servlet Filters.

Pipes and filters

In this sample we want to process a message in a sequence of steps where each steps can perform their specific function. In our example we have a JMS queue for receiving new orders. When an order is received we need to process it in several steps:

  • validate
  • register
  • send confirm email

This can be created in a route like this:

<route>
   <from uri="jms:queue:order"/>
   <pipeline>
      <bean ref="validateOrder"/>
      <bean ref="registerOrder"/>
      <bean ref="sendConfirmEmail"/>
   </pipeline>
</route>

Pipeline is default

In the route above we specify pipeline but it can be omitted as its default, so you can write the route as:

<route>
   <from uri="jms:queue:order"/>
   <bean ref="validateOrder"/>
   <bean ref="registerOrder"/>
   <bean ref="sendConfirmEmail"/>
</route>

This is commonly used not to state the pipeline.

An example where the pipeline needs to be used, is when using a multicast and "one" of the endpoints to send to (as a logical group) is a pipeline of other endpoints. For example.

<route>
   <from uri="jms:queue:order"/>
   <multicast>
     <to uri="log:org.company.log.Category"/>
     <pipeline>
       <bean ref="validateOrder"/>
       <bean ref="registerOrder"/>
       <bean ref="sendConfirmEmail"/>
     </pipeline>
   </multicast>
</route>

The above sends the order (from jms:queue:order) to two locations at the same time, our log component, and to the "pipeline" of beans which goes one to the other. If you consider the opposite, sans the <pipeline>

<route>
   <from uri="jms:queue:order"/>
   <multicast>
     <to uri="log:org.company.log.Category"/>
     <bean ref="validateOrder"/>
     <bean ref="registerOrder"/>
     <bean ref="sendConfirmEmail"/>
   </multicast>
</route>

you would see that multicast would not "flow" the message from one bean to the next, but rather send the order to all 4 endpoints (1x log, 3x bean) in parallel, which is not (for this example) what we want. We need the message to flow to the validateOrder, then to the registerOrder, then the sendConfirmEmail so adding the pipeline, provides this facility.

Where as the bean ref is a reference for a spring bean id, so we define our beans using regular Spring XML as:

   <bean id="validateOrder" class="com.mycompany.MyOrderValidator"/>

Our validator bean is a plain POJO that has no dependencies to Camel what so ever. So you can implement this POJO as you like. Camel uses rather intelligent Bean Binding to invoke your POJO with the payload of the received message. In this example we will not dig into this how this happens. You should return to this topic later when you got some hands on experience with Camel how it can easily bind routing using your existing POJO beans.

So what happens in the route above. Well when an order is received from the JMS queue the message is routed like Pipes and filters:
1. payload from the JMS is sent as input to the validateOrder bean
2. the output from validateOrder bean is sent as input to the registerOrder bean
3. the output from registerOrder bean is sent as input to the sendConfirmEmail bean

Using Camel Components

In the route lets imagine that the registration of the order has to be done by sending data to a TCP socket that could be a big mainframe. As Camel has many Components we will use the camel-mina component that supports TCP connectivity. So we change the route to:

<route>
   <from uri="jms:queue:order"/>
   <bean ref="validateOrder"/>
   <to uri="mina:tcp://mainframeip:4444?textline=true"/>
   <bean ref="sendConfirmEmail"/>
</route>

What we now have in the route is a to type that can be used as a direct replacement for the bean type. The steps is now:
1. payload from the JMS is sent as input to the validateOrder bean
2. the output from validateOrder bean is sent as text to the mainframe using TCP
3. the output from mainframe is sent back as input to the sendConfirmEmai bean

What to notice here is that the to is not the end of the route (the world (wink)) in this example it's used in the middle of the Pipes and filters. In fact we can change the bean types to to as well:

<route>
   <from uri="jms:queue:order"/>
   <to uri="bean:validateOrder"/>
   <to uri="mina:tcp://mainframeip:4444?textline=true"/>
   <to uri="bean:sendConfirmEmail"/>
</route>

As the to is a generic type we must state in the uri scheme which component it is. So we must write bean: for the Bean component that we are using.

Conclusion

This example was provided to demonstrate the Spring DSL (XML based) as opposed to the pure Java DSL from the first example. And as well to point about that the to doesn't have to be the last node in a route graph.

This example is also based on the in-only message exchange pattern. What you must understand as well is the in-out message exchange pattern, where the caller expects a response. We will look into this in another example.

See also

Getting Started with Apache Camel

The Enterprise Integration Patterns (EIP) book

The purpose of a "patterns" book is not to advocate new techniques that the authors have invented, but rather to document existing best practices within a particular field. By doing this, the authors of a patterns book hope to spread knowledge of best practices and promote a vocabulary for discussing architectural designs.
One of the most famous patterns books is Design Patterns: Elements of Reusable Object-oriented Software by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides, commonly known as the "Gang of Four" (GoF) book. Since the publication of Design Patterns, many other pattern books, of varying quality, have been written. One famous patterns book is called Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf. It is common for people to refer to this book by its initials EIP. As the subtitle of EIP suggests, the book focuses on design patterns for asynchronous messaging systems. The book discusses 65 patterns. Each pattern is given a textual name and most are also given a graphical symbol, intended to be used in architectural diagrams.

The Camel project

Camel (http://camel.apache.org) is an open-source, Java-based project that helps the user implement many of the design patterns in the EIP book. Because Camel implements many of the design patterns in the EIP book, it would be a good idea for people who work with Camel to have the EIP book as a reference.

Online documentation for Camel

The documentation is all under the Documentation category on the right-side menu of the Camel website (also available in PDF form. Camel-related books are also available, in particular the Camel in Action book, presently serving as the Camel bible--it has a free Chapter One (pdf), which is highly recommended to read to get more familiar with Camel.

A useful tip for navigating the online documentation

The breadcrumbs at the top of the online Camel documentation can help you navigate between parent and child subsections.
For example, If you are on the "Languages" documentation page then the left-hand side of the reddish bar contains the following links.

Apache Camel > Documentation > Architecture > Languages

As you might expect, clicking on "Apache Camel" takes you back to the home page of the Apache Camel project, and clicking on "Documentation" takes you to the main documentation page. You can interpret the "Architecture" and "Languages" buttons as indicating you are in the "Languages" section of the "Architecture" chapter. Adding browser bookmarks to pages that you frequently reference can also save time.

Online Javadoc documentation

The Apache Camel website provides Javadoc documentation. It is important to note that the Javadoc documentation is spread over several independent Javadoc hierarchies rather than being all contained in a single Javadoc hierarchy. In particular, there is one Javadoc hierarchy for the core APIs of Camel, and a separate Javadoc hierarchy for each component technology supported by Camel. For example, if you will be using Camel with ActiveMQ and FTP then you need to look at the Javadoc hierarchies for the core API and Spring API.

Concepts and terminology fundamental to Camel

In this section some of the concepts and terminology that are fundamental to Camel are explained. This section is not meant as a complete Camel tutorial, but as a first step in that direction.

Endpoint

The term endpoint is often used when talking about inter-process communication. For example, in client-server communication, the client is one endpoint and the server is the other endpoint. Depending on the context, an endpoint might refer to an address, such as a host:port pair for TCP-based communication, or it might refer to a software entity that is contactable at that address. For example, if somebody uses "www.example.com:80" as an example of an endpoint, they might be referring to the actual port at that host name (that is, an address), or they might be referring to the web server (that is, software contactable at that address). Often, the distinction between the address and software contactable at that address is not an important one.
Some middleware technologies make it possible for several software entities to be contactable at the same physical address. For example, CORBA is an object-oriented, remote-procedure-call (RPC) middleware standard. If a CORBA server process contains several objects then a client can communicate with any of these objects at the same physical address (host:port), but a client communicates with a particular object via that object's logical address (called an IOR in CORBA terminology), which consists of the physical address (host:port) plus an id that uniquely identifies the object within its server process. (An IOR contains some additional information that is not relevant to this present discussion.) When talking about CORBA, some people may use the term "endpoint" to refer to a CORBA server's physical address, while other people may use the term to refer to the logical address of a single CORBA object, and other people still might use the term to refer to any of the following:

  • The physical address (host:port) of the CORBA server process
  • The logical address (host:port plus id) of a CORBA object.
  • The CORBA server process (a relatively heavyweight software entity)
  • A CORBA object (a lightweight software entity)

Because of this, you can see that the term endpoint is ambiguous in at least two ways. First, it is ambiguous because it might refer to an address or to a software entity contactable at that address. Second, it is ambiguous in the granularity of what it refers to: a heavyweight versus lightweight software entity, or physical address versus logical address. It is useful to understand that different people use the term endpoint in slightly different (and hence ambiguous) ways because Camel's usage of this term might be different to whatever meaning you had previously associated with the term.
Camel provides out-of-the-box support for endpoints implemented with many different communication technologies. Here are some examples of the Camel-supported endpoint technologies.

  • A JMS queue.
  • A web service.
  • A file. A file may sound like an unlikely type of endpoint, until you realize that in some systems one application might write information to a file and, later, another application might read that file.
  • An FTP server.
  • An email address. A client can send a message to an email address, and a server can read an incoming message from a mail server.
  • A POJO (plain old Java object).

In a Camel-based application, you create (Camel wrappers around) some endpoints and connect these endpoints with routes, which I will discuss later in Section 4.8 ("Routes, RouteBuilders and Java DSL"). Camel defines a Java interface called Endpoint. Each Camel-supported endpoint has a class that implements this Endpoint interface. As I discussed in Section 3.3 ("Online Javadoc documentation"), Camel provides a separate Javadoc hierarchy for each communications technology supported by Camel. Because of this, you will find documentation on, say, the JmsEndpoint class in the JMS Javadoc hierarchy, while documentation for, say, the FtpEndpoint class is in the FTP Javadoc hierarchy.

CamelContext

A CamelContext object represents the Camel runtime system. You typically have one CamelContext object in an application. A typical application executes the following steps.

  1. Create a CamelContext object.
  2. Add endpoints – and possibly Components, which are discussed in Section 4.5 ("Components") – to the CamelContext object.
  3. Add routes to the CamelContext object to connect the endpoints.
  4. Invoke the start() operation on the CamelContext object. This starts Camel-internal threads that are used to process the sending, receiving and processing of messages in the endpoints.
  5. Eventually invoke the stop() operation on the CamelContext object. Doing this gracefully stops all the endpoints and Camel-internal threads.

Note that the CamelContext.start() operation does not block indefinitely. Rather, it starts threads internal to each Component and Endpoint and then start() returns. Conversely, CamelContext.stop() waits for all the threads internal to each Endpoint and Component to terminate and then stop() returns.
If you neglect to call CamelContext.start() in your application then messages will not be processed because internal threads will not have been created.
If you neglect to call CamelContext.stop() before terminating your application then the application may terminate in an inconsistent state. If you neglect to call CamelContext.stop() in a JUnit test then the test may fail due to messages not having had a chance to be fully processed.

CamelTemplate

Camel used to have a class called CamelClient, but this was renamed to be CamelTemplate to be similar to a naming convention used in some other open-source projects, such as the TransactionTemplate and JmsTemplate classes in Spring.
The CamelTemplate class is a thin wrapper around the CamelContext class. It has methods that send a Message or Exchange – both discussed in Section 4.6 ("Message and Exchange")) – to an Endpoint – discussed in Section 4.1 ("Endpoint"). This provides a way to enter messages into source endpoints, so that the messages will move along routes – discussed in Section 4.8 ("Routes, RouteBuilders and Java DSL") – to destination endpoints.

The Meaning of URL, URI, URN and IRI

Some Camel methods take a parameter that is a URI string. Many people know that a URI is "something like a URL" but do not properly understand the relationship between URI and URL, or indeed its relationship with other acronyms such as IRI and URN.
Most people are familiar with URLs (uniform resource locators), such as "http://...", "ftp://...", "mailto:...". Put simply, a URL specifies the location of a resource.
A URI (uniform resource identifier) is a URL or a URN. So, to fully understand what URI means, you need to first understand what is a URN.
URN is an acronym for uniform resource name. There are may "unique identifier" schemes in the world, for example, ISBNs (globally unique for books), social security numbers (unique within a country), customer numbers (unique within a company's customers database) and telephone numbers. Each "unique identifier" scheme has its own notation. A URN is a wrapper for different "unique identifier" schemes. The syntax of a URN is "urn:<scheme-name>:<unique-identifier>". A URN uniquely identifies a resource, such as a book, person or piece of equipment. By itself, a URN does not specify the location of the resource. Instead, it is assumed that a registry provides a mapping from a resource's URN to its location. The URN specification does not state what form a registry takes, but it might be a database, a server application, a wall chart or anything else that is convenient. Some hypothetical examples of URNs are "urn:employee:08765245", "urn:customer:uk:3458:hul8" and "urn:foo:0000-0000-9E59-0000-5E-2". The <scheme-name> ("employee", "customer" and "foo" in these examples) part of a URN implicitly defines how to parse and interpret the <unique-identifier> that follows it. An arbitrary URN is meaningless unless: (1) you know the semantics implied by the <scheme-name>, and (2) you have access to the registry appropriate for the <scheme-name>. A registry does not have to be public or globally accessible. For example, "urn:employee:08765245" might be meaningful only within a specific company.
To date, URNs are not (yet) as popular as URLs. For this reason, URI is widely misused as a synonym for URL.
IRI is an acronym for internationalized resource identifier. An IRI is simply an internationalized version of a URI. In particular, a URI can contain letters and digits in the US-ASCII character set, while a IRI can contain those same letters and digits, and also European accented characters, Greek letters, Chinese ideograms and so on.

Components

Component is confusing terminology; EndpointFactory would have been more appropriate because a Component is a factory for creating Endpoint instances. For example, if a Camel-based application uses several JMS queues then the application will create one instance of the JmsComponent class (which implements the Component interface), and then the application invokes the createEndpoint() operation on this JmsComponent object several times. Each invocation of JmsComponent.createEndpoint() creates an instance of the JmsEndpoint class (which implements the Endpoint interface). Actually, application-level code does not invoke Component.createEndpoint() directly. Instead, application-level code normally invokes CamelContext.getEndpoint(); internally, the CamelContext object finds the desired Component object (as I will discuss shortly) and then invokes createEndpoint() on it.
Consider the following code.

myCamelContext.getEndpoint("pop3://john.smith@mailserv.example.com?password=myPassword");

The parameter to getEndpoint() is a URI. The URI prefix (that is, the part before ":") specifies the name of a component. Internally, the CamelContext object maintains a mapping from names of components to Component objects. For the URI given in the above example, the CamelContext object would probably map the pop3 prefix to an instance of the MailComponent class. Then the CamelContext object invokes createEndpoint("pop3://john.smith@mailserv.example.com?password=myPassword") on that MailComponent object. The createEndpoint() operation splits the URI into its component parts and uses these parts to create and configure an Endpoint object.
In the previous paragraph, I mentioned that a CamelContext object maintains a mapping from component names to Component objects. This raises the question of how this map is populated with named Component objects. There are two ways of populating the map. The first way is for application-level code to invoke CamelContext.addComponent(String componentName, Component component). The example below shows a single MailComponent object being registered in the map under 3 different names.

Component mailComponent = new org.apache.camel.component.mail.MailComponent();
myCamelContext.addComponent("pop3", mailComponent);
myCamelContext.addComponent("imap", mailComponent);
myCamelContext.addComponent("smtp", mailComponent);

The second (and preferred) way to populate the map of named Component objects in the CamelContext object is to let the CamelContext object perform lazy initialization. This approach relies on developers following a convention when they write a class that implements the Component interface. I illustrate the convention by an example. Let's assume you write a class called com.example.myproject.FooComponent and you want Camel to automatically recognize this by the name "foo". To do this, you have to write a properties file called "META-INF/services/org/apache/camel/component/foo" (without a ".properties" file extension) that has a single entry in it called class, the value of which is the fully-scoped name of your class. This is shown below.

META-INF/services/org/apache/camel/component/foo
class=com.example.myproject.FooComponent

If you want Camel to also recognize the class by the name "bar" then you write another properties file in the same directory called "bar" that has the same contents. Once you have written the properties file(s), you create a jar file that contains the com.example.myproject.FooComponent class and the properties file(s), and you add this jar file to your CLASSPATH. Then, when application-level code invokes createEndpoint("foo:...") on a CamelContext object, Camel will find the "foo"" properties file on the CLASSPATH, get the value of the class property from that properties file, and use reflection APIs to create an instance of the specified class.
As I said in Section 4.1 ("Endpoint"), Camel provides out-of-the-box support for numerous communication technologies. The out-of-the-box support consists of classes that implement the Component interface plus properties files that enable a CamelContext object to populate its map of named Component objects.
Earlier in this section I gave the following example of calling CamelContext.getEndpoint().

myCamelContext.getEndpoint("pop3://john.smith@mailserv.example.com?password=myPassword");

When I originally gave that example, I said that the parameter to getEndpoint() was a URI. I said that because the online Camel documentation and the Camel source code both claim the parameter is a URI. In reality, the parameter is restricted to being a URL. This is because when Camel extracts the component name from the parameter, it looks for the first ":", which is a simplistic algorithm. To understand why, recall from Section 4.4 ("The Meaning of URL, URI, URN and IRI") that a URI can be a URL or a URN. Now consider the following calls to getEndpoint.

myCamelContext.getEndpoint("pop3:...");
myCamelContext.getEndpoint("jms:...");
myCamelContext.getEndpoint("urn:foo:...");
myCamelContext.getEndpoint("urn:bar:...");

Camel identifies the components in the above example as "pop3", "jms", "urn" and "urn". It would be more useful if the latter components were identified as "urn:foo" and "urn:bar" or, alternatively, as "foo" and "bar" (that is, by skipping over the "urn:" prefix). So, in practice you must identify an endpoint with a URL (a string of the form "<scheme>:...") rather than with a URN (a string of the form "urn:<scheme>:..."). This lack of proper support for URNs means the you should consider the parameter to getEndpoint() as being a URL rather than (as claimed) a URI.

Message and Exchange

The Message interface provides an abstraction for a single message, such as a request, reply or exception message.
There are concrete classes that implement the Message interface for each Camel-supported communications technology. For example, the JmsMessage class provides a JMS-specific implementation of the Message interface. The public API of the Message interface provides get- and set-style methods to access the message id, body and individual header fields of a messge.
The Exchange interface provides an abstraction for an exchange of messages, that is, a request message and its corresponding reply or exception message. In Camel terminology, the request, reply and exception messages are called in, out and fault messages.
There are concrete classes that implement the Exchange interface for each Camel-supported communications technology. For example, the JmsExchange class provides a JMS-specific implementation of the Exchange interface. The public API of the Exchange interface is quite limited. This is intentional, and it is expected that each class that implements this interface will provide its own technology-specific operations.
Application-level programmers rarely access the Exchange interface (or classes that implement it) directly. However, many classes in Camel are generic types that are instantiated on (a class that implements) Exchange. Because of this, the Exchange interface appears a lot in the generic signatures of classes and methods.

Processor

The Processor interface represents a class that processes a message. The signature of this interface is shown below.

Processor
package org.apache.camel;
public interface Processor {
    void process(Exchange exchange) throws Exception;
}

Notice that the parameter to the process() method is an Exchange rather than a Message. This provides flexibility. For example, an implementation of this method initially might call exchange.getIn() to get the input message and process it. If an error occurs during processing then the method can call exchange.setException().
An application-level developer might implement the Processor interface with a class that executes some business logic. However, there are many classes in the Camel library that implement the Processor interface in a way that provides support for a design pattern in the EIP book. For example, ChoiceProcessor implements the message router pattern, that is, it uses a cascading if-then-else statement to route a message from an input queue to one of several output queues. Another example is the FilterProcessor class which discards messages that do not satisfy a stated predicate (that is, condition).

Routes, RouteBuilders and Java DSL

A route is the step-by-step movement of a Message from an input queue, through arbitrary types of decision making (such as filters and routers) to a destination queue (if any). Camel provides two ways for an application developer to specify routes. One way is to specify route information in an XML file. A discussion of that approach is outside the scope of this document. The other way is through what Camel calls a Java DSL (domain-specific language).

Introduction to Java DSL

For many people, the term "domain-specific language" implies a compiler or interpreter that can process an input file containing keywords and syntax specific to a particular domain. This is not the approach taken by Camel. Camel documentation consistently uses the term "Java DSL" instead of "DSL", but this does not entirely avoid potential confusion. The Camel "Java DSL" is a class library that can be used in a way that looks almost like a DSL, except that it has a bit of Java syntactic baggage. You can see this in the example below. Comments afterwards explain some of the constructs used in the example.

Example of Camel's "Java DSL"
RouteBuilder builder = new RouteBuilder() {
    public void configure() {
        from("queue:a").filter(header("foo").isEqualTo("bar")).to("queue:b");
        from("queue:c").choice()
                .when(header("foo").isEqualTo("bar")).to("queue:d")
                .when(header("foo").isEqualTo("cheese")).to("queue:e")
                .otherwise().to("queue:f");
    }
};
CamelContext myCamelContext = new DefaultCamelContext();
myCamelContext.addRoutes(builder);

The first line in the above example creates an object which is an instance of an anonymous subclass of RouteBuilder with the specified configure() method.
The CamelContext.addRoutes(RouterBuilder builder) method invokes builder.setContext(this) – so the RouteBuilder object knows which CamelContext object it is associated with – and then invokes builder.configure(). The body of configure() invokes methods such as from(), filter(), choice(), when(), isEqualTo(), otherwise() and to().
The RouteBuilder.from(String uri) method invokes getEndpoint(uri) on the CamelContext associated with the RouteBuilder object to get the specified Endpoint and then puts a FromBuilder "wrapper" around this Endpoint. The FromBuilder.filter(Predicate predicate) method creates a FilterProcessor object for the Predicate (that is, condition) object built from the header("foo").isEqualTo("bar") expression. In this way, these operations incrementally build up a Route object (with a RouteBuilder wrapper around it) and add it to the CamelContext object associated with the RouteBuilder.

Critique of Java DSL

The online Camel documentation compares Java DSL favourably against the alternative of configuring routes and endpoints in a XML-based Spring configuration file. In particular, Java DSL is less verbose than its XML counterpart. In addition, many integrated development environments (IDEs) provide an auto-completion feature in their editors. This auto-completion feature works with Java DSL, thereby making it easier for developers to write Java DSL.
However, there is another option that the Camel documentation neglects to consider: that of writing a parser that can process DSL stored in, say, an external file. Currently, Camel does not provide such a DSL parser, and I do not know if it is on the "to do" list of the Camel maintainers. I think that a DSL parser would offer a significant benefit over the current Java DSL. In particular, the DSL would have a syntactic definition that could be expressed in a relatively short BNF form. The effort required by a Camel user to learn how to use DSL by reading this BNF would almost certainly be significantly less than the effort currently required to study the API of the RouterBuilder classes.

Continue Learning about Camel

Return to the main Getting Started page for additional introductory reference information.

Architecture

Camel uses a Java based Routing Domain Specific Language (DSL) or an Xml Configuration to configure routing and mediation rules which are added to a CamelContext to implement the various Enterprise Integration Patterns.

At a high level Camel consists of a CamelContext which contains a collection of Component instances. A Component is essentially a factory of Endpoint instances. You can explicitly configure Component instances in Java code or an IoC container like Spring or Guice, or they can be auto-discovered using URIs.

An Endpoint acts rather like a URI or URL in a web application or a Destination in a JMS system; you can communicate with an endpoint; either sending messages to it or consuming messages from it. You can then create a Producer or Consumer on an Endpoint to exchange messages with it.

The DSL makes heavy use of pluggable Languages to create an Expression or Predicate to make a truly powerful DSL which is extensible to the most suitable language depending on your needs. The following languages are supported

For a full details of the individual languages see the Language Appendix

URIs

Camel makes extensive use of URIs to allow you to refer to endpoints which are lazily created by a Component if you refer to them within Routes.

important

Make sure to read How do I configure endpoints to learn more about configuring endpoints. For example how to refer to beans in the Registry or how to use raw values for password options, and using property placeholders etc.

Current Supported URIs

Component / ArtifactId / URI

Description

AHCcamel-ahc

ahc:http[s]://hostName[:port][/resourceUri][?options]

To call external HTTP services using Async Http Client

AHC-WS camel-ahc-ws

ahc-ws[s]://hostName[:port][/resourceUri][?options]


 To exchange data with external Websocket servers using Async Http Client

AMQPcamel-amqp

amqp:[queue:|topic:]destinationName[?options]

For Messaging with AMQP protocol

APNScamel-apns

apns:<notify|consumer>[?options]

For sending notifications to Apple iOS devices

Atmosphere-Websocket   camel-atmosphere-websocket

atmosphere-websocket:///relative path[?options]


 To exchange data with external Websocket clients using Atmosphere

Atomcamel-atom

atom:atomUri[?options]

Working with Apache Abdera for atom integration, such as consuming an atom feed.

Avrocamel-avro

avro:[transport]:[host]:[port][/messageName][?options]

Working with Apache Avro for data serialization.

AWS-CW / camel-aws

aws-cw://namespace[?options]

For working with Amazon's CloudWatch (CW).

AWS-DDB / camel-aws

aws-ddb://tableName[?options]

For working with Amazon's DynamoDB (DDB).

AWS-DDBSTREAM / camel-aws

aws-ddbstream://tableName[?options]

For working with Amazon's DynamoDB Streams (DDB Streams).

AWS-EC2 / camel-aws

aws-ec2://label[?options]

For working with Amazon's Elastic Compute Cloud (EC2).

AWS-SDB / camel-aws

aws-sdb://domainName[?options]

For working with Amazon's SimpleDB (SDB).

AWS-SES / camel-aws

aws-ses://from[?options]

For working with Amazon's Simple Email Service (SES).

AWS-SNS / camel-aws

aws-sns://topicName[?options]

For Messaging with Amazon's Simple Notification Service (SNS).

AWS-SQS / camel-aws

aws-sqs://queueName[?options]

For Messaging with Amazon's Simple Queue Service (SQS).

AWS-SWF / camel-aws

aws-swf://<worfklow|activity>[?options]

For Messaging with Amazon's Simple Workflow Service (SWF).

AWS-S3 / camel-aws

aws-s3://bucketName[?options]

For working with Amazon's Simple Storage Service (S3).

Beancamel-core

bean:beanName[?options]

Uses the Bean Binding to bind message exchanges to beans in the Registry. Is also used for exposing and invoking POJO (Plain Old Java Objects).

Beanstalk camel-beanstalk

beanstalk:hostname:port/tube[?options]

For working with Amazon's Beanstalk.

Bean Validatorcamel-bean-validator

bean-validator:label[?options]

Validates the payload of a message using the Java Validation API (JSR 303 and JAXP Validation) and its reference implementation Hibernate Validator

Boxcamel-box

box://endpoint-prefix/endpoint?[options]

For uploading, downloading and managing files, managing files, folders, groups, collaborations, etc. on Box.com.

Braintreecamel-braintree

braintree://endpoint-prefix/endpoint?[options]


Component for interacting with Braintree Payments via Braintree Java SDK

Browsecamel-core

browse:someName

Provides a simple BrowsableEndpoint which can be useful for testing, visualisation tools or debugging. The exchanges sent to the endpoint are all available to be browsed.

Cachecamel-cache

cache://cacheName[?options]

The cache component facilitates creation of caching endpoints and processors using EHCache as the cache implementation.

Cassandra / camel-cassandraql

cql:localhost/keyspace


For integrating with Apache Cassandra.

Classcamel-core

class:className[?options]

Uses the Bean Binding to bind message exchanges to beans in the Registry. Is also used for exposing and invoking POJO (Plain Old Java Objects).

Chronicle Enginecamel-chronicle

chronicle-engine:addresses/path[?options]
Chronicle Engine is a high performance, low latency, reactive processing framework.

Chunkcamel-chunk

chunk:templateName[?options]

Generates a response using a Chunk template

CMIScamel-cmis

cmis://cmisServerUrl[?options]

Uses the Apache Chemistry client API to interface with CMIS supporting CMS

Cometdcamel-cometd

cometd://hostName:port/channelName[?options]

Used to deliver messages using the jetty cometd implementation of the bayeux protocol

Consulcamel-consul

consul:apiEndpoint[?options]

For interfacing with an  Consul.

Contextcamel-context

context:camelContextId:localEndpointName[?options]

Used to refer to endpoints within a separate CamelContext to provide a simple black box composition approach so that routes can be combined into a CamelContext and then used as a black box component inside other routes in other CamelContexts

ControlBuscamel-core

controlbus:command[?options]

ControlBus EIP that allows to send messages to Endpoints for managing and monitoring your Camel applications.

CouchDBcamel-couchdb

couchdb:hostName[:port]/database[?options]

To integrate with Apache CouchDB.

Crypto (Digital Signatures)camel-crypto

crypto:<sign|verify>:name[?options]

Used to sign and verify exchanges using the Signature Service of the Java Cryptographic Extension.

CXFcamel-cxf

cxf:<bean:cxfEndpoint|//someAddress>[?options]

Working with Apache CXF for web services integration

CXF Bean camel-cxf

cxfbean:serviceBeanRef[?options]

Proceess the exchange using a JAX WS or JAX RS annotated bean from the registry. Requires less configuration than the above CXF Component

CXFRScamel-cxf

cxfrs:<bean:rsEndpoint|//address>[?options]

Working with Apache CXF for REST services integration

DataFormatcamel-core

dataformat:name:<marshal|unmarshal>[?options]

for working with Data Formats as if it was a regular Component supporting Endpoints and URIs.

DataSetcamel-core

dataset:name[?options]

For load & soak testing the DataSet provides a way to create huge numbers of messages for sending to Components or asserting that they are consumed correctly

Directcamel-core

direct:someName[?options]

Synchronous call to another endpoint from same CamelContext.

Direct-VMcamel-core

direct-vm:someName[?options]

Synchronous call to another endpoint in another CamelContext running in the same JVM.

DNScamel-dns

dns:operation[?options]

To lookup domain information and run DNS queries using DNSJava

Disruptorcamel-disruptor

disruptor:someName[?<option>]
disruptor-vm:someName[?<option>]

To provide the implementation of SEDA which is based on disruptor

Dockercamel-docker

docker://[operation]?[options]


 To communicate with Docker

Dozercamel-dozer

dozer://name?[options]

 To convert message body using the Dozer type converter library.

Dropbox camel-dropbox

dropbox://[operation]?[options]

The  dropbox:  component allows you to treat  Dropbox  remote folders as a producer or consumer of messages.

EJBcamel-ejb

ejb:ejbName[?options]

Uses the Bean Binding to bind message exchanges to EJBs. It works like the Bean component but just for accessing EJBs. Supports EJB 3.0 onwards.

Ehcachecamel-ehcache

ehcache://cacheName[?options]

The cache component facilitates creation of caching endpoints and processors using Ehcache 3 as the cache implementation.

ElasticSearchcamel-elasticsearch

elasticsearch://clusterName[?options]

For interfacing with an ElasticSearch server.

Etcdcamel-etcd

etcd:namespace[/path][?options]

For interfacing with an Etcd key value store.

Spring Eventcamel-spring

spring-event://default

Working with Spring ApplicationEvents

EventAdmincamel-eventadmin

eventadmin:topic[?options]

Receiving OSGi EventAdmin events

Execcamel-exec

exec://executable[?options]

For executing system commands

Facebookcamel-facebook

facebook://endpoint[?options]

Providing access to all of the Facebook APIs accessible using Facebook4J

Filecamel-core

file://nameOfFileOrDirectory[?options]

Sending messages to a file or polling a file or directory.

Flatpackcamel-flatpack

flatpack:[fixed|delim]:configFile[?options]

Processing fixed width or delimited files or messages using the FlatPack library

Flinkcamel-flink

flink:dataset[?options]
flink:datastream[?options]

 Bridges Camel connectors with Apache Flink tasks.

FOPcamel-fop

fop:outputFormat[?options]

Renders the message into different output formats using Apache FOP

FreeMarkercamel-freemarker

freemarker:templateName[?options]

Generates a response using a FreeMarker template

FTPcamel-ftp

ftp:contextPath[?options]

Sending and receiving files over FTP.

FTPScamel-ftp

ftps://[username@]hostName[:port]/directoryName[?options]

Sending and receiving files over FTP Secure (TLS and SSL).

Gangliacamel-ganglia

ganglia:destination:port[?options]

Sends values as metrics to the Ganglia performance monitoring system using gmetric4j.  Can be used along with JMXetric.

GAuth / camel-gae

gauth://name[?options]

Used by web applications to implement an OAuth consumer. See also Camel Components for Google App Engine.

GHttp / camel-gae

ghttp:contextPath[?options]

Provides connectivity to the URL fetch service of Google App Engine but can also be used to receive messages from servlets. See also Camel Components for Google App Engine.

Git / camel-git

git:localRepositoryPath[?options]

Supports interaction with Git repositories

Github / camel-github

github:endpoint[?options]

Supports interaction with Github

GLogin / camel-gae

glogin://hostname[:port][?options]

Used by Camel applications outside Google App Engine (GAE) for programmatic login to GAE applications. See also Camel Components for Google App Engine.

GTask / camel-gae

gtask://queue-name[?options]

Supports asynchronous message processing on Google App Engine by using the task queueing service as message queue. See also Camel Components for Google App Engine.

Google Calendar / camel-google-calendar

google-calendar://endpoint-prefix/endpoint?[options] 

Supports interaction with Google Calendar's REST API.

Google Drive / camel-google-drive

google-drive://endpoint-prefix/endpoint?[options]

Supports interaction with Google Drive's REST API.

Google Mail / camel-google-mail

google-mail://endpoint-prefix/endpoint?[options]

Supports interaction with Google Mail's REST API.

GMail / camel-gae

gmail://user@g[oogle]mail.com[?options]

Supports sending of emails via the mail service of Google App Engine. See also Camel Components for Google App Engine.

Gora camel-gora

gora:instanceName[?options]


Supports to work with NoSQL databases using the Apache Gora framework.

Grapecamel-grape

 grape:defaultMavenCoordinates

Grape component allows you to fetch, load and manage additional jars when CamelContext is running.

Geocodercamel-geocoder

geocoder:<address|latlng:latitude,longitude>[?options]

Supports looking up geocoders for an address, or reverse lookup geocoders from an address.

Google Guava EventBuscamel-guava-eventbus

guava-eventbus:busName[?options]

The Google Guava EventBus allows publish-subscribe-style communication between components without requiring the components to explicitly register with one another (and thus be aware of each other). This component provides integration bridge between Camel and Google Guava EventBus infrastructure.

Hazelcast / camel-hazelcast

hazelcast://[type]:cachename[?options]

Hazelcast is a data grid entirely implemented in Java (single jar). This component supports map, multimap, seda, queue, set, atomic number and simple cluster support.

HBasecamel-hbase

hbase://table[?options]

For reading/writing from/to an HBase store (Hadoop database)

HDFScamel-hdfs

hdfs://hostName[:port][/path][?options]

For reading/writing from/to an HDFS filesystem using Hadoop 1.x

HDFS2camel-hdfs2

hdfs2://hostName[:port][/path][?options]

For reading/writing from/to an HDFS filesystem using Hadoop 2.x

Hipchatcamel-hipchat

hipchat://[host][:port]?options

 For sending/receiving messages to Hipchat using v2 API

HL7camel-hl7

mina2:tcp://hostName[:port][?options]

For working with the HL7 MLLP protocol and the HL7 data format using the HAPI library

Infinispancamel-infinispan

infinispan://cacheName[?options]

For reading/writing from/to Infinispan distributed key/value store and data grid

HTTPcamel-http

http:hostName[:port][/resourceUri][?options]

For calling out to external HTTP servers using Apache HTTP Client 3.x

HTTP4camel-http4

http4:hostName[:port][/resourceUri][?options]

For calling out to external HTTP servers using Apache HTTP Client 4.x

iBATIScamel-ibatis

ibatis://statementName[?options]

Performs a query, poll, insert, update or delete in a relational database using Apache iBATIS

Ignitecamel-ignite

ignite:[cache/compute/messaging/...][?options]

Apache Ignite  In-Memory Data Fabric is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. It is designed to deliver uncompromised performance for a wide set of in-memory computing use cases from high performance computing, to the industry most advanced data grid, highly available service grid, and streaming.

IMAPcamel-mail

imap://[username@]hostName[:port][?options]

Receiving email using IMAP

IMAPScamel-mail

imaps://[username@]hostName[:port][?options]

...

IRCcamel-irc

irc:[login@]hostName[:port]/#room[?options]

For IRC communication

IronMQ camel-ironmq

ironmq:queueName[?options]


For working with IronMQ a elastic and durable hosted message queue as a service.

JavaSpacecamel-javaspace

javaspace:jini://hostName[?options]

Sending and receiving messages through JavaSpace

jBPMcamel-jbpm

jbpm:hostName[:port][/resourceUri][?options]

Sending messages through kie-remote-client API to jBPM.

jcachecamel-jcache

jcache:cacheName[?options]

The JCache component facilitates creation of caching endpoints and processors using JCache / jsr107 as the cache implementation.

jcloudscamel-jclouds

jclouds:<blobstore|compute>:[provider id][?options]

For interacting with cloud compute & blobstore service via jclouds

JCRcamel-jcr

jcr://user:password@repository/path/to/node[?options]

Storing a message in a JCR compliant repository like Apache Jackrabbit

JDBCcamel-jdbc

jdbc:dataSourceName[?options]

For performing JDBC queries and operations

Jettycamel-jetty

jetty:hostName[:port][/resourceUri][?options]

For exposing or consuming services over HTTP

JGroupscamel-jgroups

jgroups:clusterName[?options]

The jgroups: component provides exchange of messages between Camel infrastructure and JGroups clusters.

JIRAcamel-jira

jira://endpoint[?options]

For interacting with JIRA

JMScamel-jms

jms:[queue:|topic:]destinationName[?options]

Working with JMS providers

JMXcamel-jmx

jmx://platform[?options]

For working with JMX notification listeners

JPAcamel-jpa

jpa://entityName[?options]

For using a database as a queue via the JPA specification for working with OpenJPA, Hibernate or TopLink

JOLT camel-jolt

jolt:specName[?options]


 

The jolt: component allows you to process a JSON messages using an JOLT specification. This can be ideal when doing JSON to JSON transformation.

Jschcamel-jsch

scp://hostName[:port]/destination[?options]

Support for the scp protocol

JT/400 camel-jt400

jt400://user:pwd@system/<path_to_dtaq>[?options]

For integrating with data queues on an AS/400 (aka System i, IBM i, i5, ...) system

Kafkacamel-kafka

kafka://server:port[?options]


For producing to or consuming from Apache Kafka message brokers.

Kestrelcamel-kestrel

kestrel://[addresslist/]queueName[?options]

For producing to or consuming from Kestrel queues

Kraticamel-krati

krati://[path to datastore/][?options]

For producing to or consuming to Krati datastores

Kubernetescamel-kubernetes

kubernetes:masterUrl[?options]

 For integrating your application with Kubernetes standalone or on top of OpenShift.

Kuracamel-kura

 

For deploying Camel OSGi routes into the Eclipse Kura M2M container.

Languagecamel-core

language://languageName[:script][?options]

Executes Languages scripts

LDAPcamel-ldap

ldap:host[:port][?options]

Performing searches on LDAP servers (<scope> must be one of object|onelevel|subtree)

LinkedIncamel-linkedin

linkedin://endpoint-prefix/endpoint?[options]

Component for retrieving LinkedIn user profiles, connections, companies, groups, posts, etc. using LinkedIn REST API.

Logcamel-core

log:loggingCategory[?options]

Uses Jakarta Commons Logging to log the message exchange to some underlying logging system like log4j

Lucenecamel-lucene

lucene:searcherName:<insert|query>[?options]

Uses Apache Lucene to perform Java-based indexing and full text based searches using advanced analysis/tokenization capabilities

Lumberjackcamel-lumberjack

lumberjack:host[:port]

 Uses the Lumberjack protocol for retrieving logs (from Filebeat for instance)

Metricscamel-metrics

metrics:[meter|counter|histogram|timer]:metricname[?options]

Uses Metrics   to collect application statistics directly from Camel routes.

MINAcamel-mina

mina:[tcp|udp|vm]:host[:port][?options]

Working with Apache MINA 1.x

MINA2camel-mina2

mina2:[tcp|udp|vm]:host[:port][?options]

Working with Apache MINA 2.x

Mockcamel-core

mock:name[?options]

For testing routes and mediation rules using mocks

MLLPcamel-mllp

mllp:host:port[?options]


The MLLP component is specifically designed to handle the nuances of the MLLP protocol and provide the functionality required by Healthcare providers to communicate with other systems using the MLLP protocol

MongoDBcamel-mongodb

mongodb:connectionBean[?options]

Interacts with MongoDB databases and collections. Offers producer endpoints to perform CRUD-style operations and more against databases and collections, as well as consumer endpoints to listen on collections and dispatch objects to Camel routes

MongoDB GridFScamel-mongodb-gridfs

mongodb-gridfs:dbName[?options]

Sending and receiving files via MongoDB's GridFS system. Note: for Camel < 2.19, the URI syntax is gridfs:dbName[?options]

MQTTcamel-mqtt

mqtt:name[?options]

Component for communicating with MQTT M2M message brokers

MSVcamel-msv

msv:someLocalOrRemoteResource[?options]

Validates the payload of a message using the MSV Library

Mustachecamel-mustache

mustache:templateName[?options]

Generates a response using a Mustache template

MVELcamel-mvel

mvel:templateName[?options]

Generates a response using an MVEL template

MyBatiscamel-mybatis

mybatis://statementName[?options]

Performs a query, poll, insert, update or delete in a relational database using MyBatis

Nagioscamel-nagios

nagios://hostName[:port][?options]

Sending passive checks to Nagios using JSendNSCA

NATScamel-nats

nats://servers[?options] 

For messaging with the NATS platform.

Nettycamel-netty

netty:<tcp|udp>//host[:port][?options]

Working with TCP and UDP protocols using Java NIO based capabilities offered by the Netty project

Netty4 camel-netty4

netty4:<tcp|udp>//host[:port][?options]


 Working with TCP and UDP protocols using Java NIO based capabilities offered by the Netty project

Netty HTTPcamel-netty-http

netty-http:http:[port]/context-path[?options]

Netty HTTP server and client using the Netty project

Netty4 HTTPcamel-netty4-http

netty4-http:http:[port]/context-path[?options]
 Netty HTTP server and client using the Netty project 4.x

Olingo2camel-olingo2

olingo2:endpoint/resource-path[?options]

Communicates with OData 2.0 services using Apache Olingo 2.0.

Openshiftcamel-openshift

openshift:clientId[?options]

To manage your Openshift applications.

OptaPlannercamel-optaplanner

optaplanner:solverConfig[?options]

Solves the planning problem contained in a message with OptaPlanner.

Paho camel-paho

paho:topic[?options]


 Paho component provides connector for the MQTT messaging protocol using the Paho library.

Pax-Loggingcamel-paxlogging

paxlogging:appender

Receiving Pax-Logging events in OSGi

PDFcamel-pdf

pdf:operation[?options]

Allows to work with Apache PDFBox PDF documents

PGEvent camel-pgevent

pgevent:dataSource[?options]


Allows for Producing/Consuming PostgreSQL events related to the LISTEN/NOTIFY commands added since PostgreSQL 8.3

POP3camel-mail

pop3s://[username@]hostName port][?options]

Receiving email using POP3 and JavaMail

POP3Scamel-mail

pop3s://[username@]hostName port][?options]

...

Printercamel-printer

lpr://host:port/path/to/printer[?options]

The printer component facilitates creation of printer endpoints to local, remote and wireless printers. The endpoints provide the ability to print camel directed payloads when utilized on camel routes.

Propertiescamel-core

properties://key[?options]

The properties component facilitates using property placeholders directly in endpoint URI definitions.

Quartzcamel-quartz

quartz://groupName/timerName[?options]

Provides a scheduled delivery of messages using the Quartz 1.x scheduler

Quartz2camel-quartz2

quartz2://groupName/timerName[?options]

Provides a scheduled delivery of messages using the Quartz 2.x scheduler

Quickfixcamel-quickfix

quickfix:configFile[?options]

Implementation of the QuickFix for Java engine which allow to send/receive FIX messages

RabbitMQcamel-rabbitmq

rabbitmq://hostname[:port]/exchangeName[?options]

Component for integrating with RabbitMQ

Refcamel-core

ref:name

Component for lookup of existing endpoints bound in the Registry.

Restcamel-core

rest:verb:path[?options]

Component for consuming Restful resources supporting the Rest DSL and plugins to other Camel rest components.

Restletcamel-restlet

restlet:restletUrl[?options]

Component for consuming and producing Restful resources using Restlet

REST Swagger / camel-rest-swagger

rest-swagger:[specificationUri#]operationId[?options]

Component for accessing REST resources using Swagger specification as configuration.

RMIcamel-rmi

rmi://hostName[:port][?options]

Working with RMI

RNCcamel-jing

rnc:/relativeOrAbsoluteUri[?options]

Validates the payload of a message using RelaxNG Compact Syntax

RNGcamel-jing

rng:/relativeOrAbsoluteUri[?options]

Validates the payload of a message using RelaxNG

Routeboxcamel-routebox

routebox:routeBoxName[?options]

Facilitates the creation of specialized endpoints that offer encapsulation and a strategy/map based indirection service to a collection of camel routes hosted in an automatically created or user injected camel context

RSScamel-rss

rss:uri[?options]

Working with ROME for RSS integration, such as consuming an RSS feed.

Salesforcecamel-salesforce

salesforce:topic[?options]

To integrate with Salesforce

SAP NetWeavercamel-sap-netweaver

sap-netweaver:hostName[:port][?options]

To integrate with SAP NetWeaver Gateway

Schedulercamel-core

scheduler://name?[options]

Used to generate message exchanges when a scheduler fires. The scheduler has more functionality than the timer component.

schematroncamel-schematron

schematron://path?[options]

Camel component of Schematron which supports to validate the XML instance documents.

SEDAcamel-core

seda:someName[?options]

Asynchronous call to another endpoint in the same CamelContext

ServiceNowcamel-servicenow

servicenow:instanceName[?options]

 Camel component for ServiceNow

SERVLETcamel-servlet

servlet:relativePath[?options]

For exposing services over HTTP through the servlet which is deployed into the Web container.

SFTPcamel-ftp

sftp://[username@]hostName[:port]/directoryName[?options]

Sending and receiving files over SFTP (FTP over SSH).

Sipcamel-sip

sip://user@hostName[:port][?options]

Publish/Subscribe communication capability using the Telecom SIP protocol. RFC3903 - Session Initiation Protocol (SIP) Extension for Event

SIPScamel-sip

sips://user@hostName[:port][?options]

...

SJMS  / camel-sjms

sjms:[queue:|topic:]destinationName[?options]

A ground up implementation of a JMS client

sjms-batch:[queue:]destinationName[?options]

A specialized JMS component for highly-performant transactional batch consumption from a queue.

Slackcamel-slack

slack:#channel[?options]

 The  slack  component allows you to connect to an instance of  Slack  and delivers a message contained in the message body via a pre established  Slack incoming webhook .

SMTPcamel-mail

smtps://[username@]hostName[:port][?options]

Sending email using SMTP and JavaMail

SMTPcamel-mail

smtps://[username@]hostName[:port][?options]

...

SMPPcamel-smpp

smpp://[username@]hostName[:port][?options]

To send and receive SMS using Short Messaging Service Center using the JSMPP library

SMPPScamel-smpp

smpps://[username@]hostName[:port][?options]

...

SNMPcamel-snmp

snmp://hostName[:port][?options]

Polling OID values and receiving traps using SNMP via SNMP4J library

Solrcamel-solr

solr://hostName[:port]/solr[?options]

Uses the Solrj client API to interface with an Apache Lucene Solr server

Apache Sparkcamel-spark

spark:{rdd|dataframe|hive}[?options]

Bridges Apache Spark computations with Camel endpoints.

Spark-restcamel-spark-rest

spark-rest://verb:path[?options]


 For easily defining REST services endpoints using Spark REST Java library.

Splunkcamel-splunk

splunk://[endpoint][?options]

For working with Splunk

SpringBatchcamel-spring-batch

spring-batch://jobName[?options]

To bridge Camel and Spring Batch

SpringIntegrationcamel-spring-integration

spring-integration:defaultChannelName[?options]

The bridge component of Camel and Spring Integration

Spring LDAPcamel-spring-ldap

spring-ldap:springLdapTemplateBean[?options]

Camel wrapper for Spring LDAP

Spring Rediscamel-spring-redis

spring-redis://hostName:port[?options]

Component for consuming and producing from Redis key-value store Redis

Spring Web Servicescamel-spring-ws

spring-ws:[mapping-type:]address[?options]

Client-side support for accessing web services, and server-side support for creating your own contract-first web services using Spring Web Services

SQLcamel-sql

sql:select * from table where id=#[?options]

Performing SQL queries using JDBC

SQL Stored Procedure camel-sql

sql-stored:template[?options]


Performing SQL queries using Stored Procedure calls

SSH component / camel-ssh

ssh:[username[:password]@]hostName[:port][?options]

For sending commands to a SSH server

StAXcamel-stax

stax:(contentHandlerClassName|#myHandler)

Process messages through a SAX ContentHandler.

Streamcamel-stream

stream:[in|out|err|file|header|url][?options]

Read or write to an input/output/error/file stream rather like unix pipes

Stompcamel-stomp

stomp:queue:destinationName[?options]

For communicating with Stomp compliant message brokers, like Apache ActiveMQ or ActiveMQ Apollo

StringTemplatecamel-stringtemplate

string-template:templateName[?options]

Generates a response using a String Template

Stubcamel-core

stub:someOtherCamelUri[?options]

Allows you to stub out some physical middleware endpoint for easier testing or debugging

Telegramcamel-telegram

telegram://bots/authToken[?options]

Allows to exchange data with the Telegram messaging network

Testcamel-spring

test:expectedMessagesEndpointUri[?options]

Creates a Mock endpoint which expects to receive all the message bodies that could be polled from the given underlying endpoint

Timercamel-core

timer:timerName[?options]

Used to generate message exchanges when a timer fires You can only consume events from this endpoint.

Twittercamel-twitter

twitter://endpoint[?options]

A twitter endpoint

Undertowcamel-undertow

undertow://host:port/context-path[?options]

HTTP server and client using the light-weight Undertow server.

Validationcamel-core (camel-spring for Camel 2.8 or older)

validation:someLocalOrRemoteResource[?options]

Validates the payload of a message using XML Schema and JAXP Validation

Velocitycamel-velocity

velocity:templateName[?options]

Generates a response using an Apache Velocity template

Vertxcamel-vertx

vertx:eventBusName

Working with the vertx event bus

VMcamel-core

vm:queueName[?options]

Asynchronous call to another endpoint in the same JVM

Weathercamel-weather

wweather://name[?options]

Polls the weather information from Open Weather Map

Websocketcamel-websocket

websocket://hostname[:port][/resourceUri][?options]

Communicating with Websocket clients

XML Security camel-xmlsecurity

xmlsecurity:<sign|verify>:name[?options]

Used to sign and verify exchanges using the XML signature specification.

XMPPcamel-xmpp

xmpp://[login@]hostname[:port][/participant][?options]

Working with XMPP and Jabber

XQuerycamel-saxon

xquery:someXQueryResource

Generates a response using an XQuery template

XSLTcamel-core (camel-spring for Camel 2.8 or older)

xslt:templateName[?options]

Generates a response using an XSLT template

Yammercamel-yammer

yammer://function[?options]

Allows you to interact with the Yammer enterprise social network

Zookeepercamel-zookeeper

zookeeper://zookeeperServer[:port][/path][?options]

Working with ZooKeeper cluster(s)

 





URI's for external components

Other projects and companies have also created Camel components to integrate additional functionality into Camel. These components may be provided under licenses that are not compatible with the Apache License, use libraries that are not compatible, etc... These components are not supported by the Camel team, but we provide links here to help users find the additional functionality.

Error rendering macro 'include'

org.owasp.validator.html.ScanException: java.util.EmptyStackException

For a full details of the individual components see the Component Appendix

Enterprise Integration Patterns

Camel supports most of the Enterprise Integration Patterns from the excellent book of the same name by Gregor Hohpe and Bobby Woolf. Its a highly recommended book, particularly for users of Camel.

Pattern Index

There now follows a list of the Enterprise Integration Patterns from the book along with examples of the various patterns using Apache Camel

Messaging Systems

Message Channel

How does one application communicate with another using messaging?

Message

How can two applications connected by a message channel exchange a piece of information?

Pipes and Filters

How can we perform complex processing on a message while maintaining independence and flexibility?

Message Router

How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions?

Message Translator

How can systems using different data formats communicate with each other using messaging?

Message Endpoint

How does an application connect to a messaging channel to send and receive messages?

Messaging Channels

Point to Point Channel

How can the caller be sure that exactly one receiver will receive the document or perform the call?

Publish Subscribe Channel

How can the sender broadcast an event to all interested receivers?

Dead Letter Channel

What will the messaging system do with a message it cannot deliver?

Guaranteed Delivery

How can the sender make sure that a message will be delivered, even if the messaging system fails?

Message Bus

What is an architecture that enables separate applications to work together, but in a de-coupled fashion such that applications can be easily added or removed without affecting the others?

Message Construction

Event Message

How can messaging be used to transmit events from one application to another?

Request Reply

When an application sends a message, how can it get a response from the receiver?

Correlation Identifier

How does a requestor that has received a reply know which request this is the reply for?

Return Address

How does a replier know where to send the reply?

Message Routing

Content Based Router

How do we handle a situation where the implementation of a single logical function (e.g., inventory check) is spread across multiple physical systems?

Message Filter

How can a component avoid receiving uninteresting messages?

Dynamic Router

How can you avoid the dependency of the router on all possible destinations while maintaining its efficiency?

Recipient List

How do we route a message to a list of (static or dynamically) specified recipients?

Splitter

How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?

Aggregator

How do we combine the results of individual, but related messages so that they can be processed as a whole?

Resequencer

How can we get a stream of related but out-of-sequence messages back into the correct order?

Composed Message Processor

How can you maintain the overall message flow when processing a message consisting of multiple elements, each of which may require different processing?

Scatter-Gather

How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?

Routing Slip

How do we route a message consecutively through a series of processing steps when the sequence of steps is not known at design-time and may vary for each message?

Throttler

How can I throttle messages to ensure that a specific endpoint does not get overloaded, or we don't exceed an agreed SLA with some external service?

Sampling

How can I sample one message out of many in a given period to avoid downstream route does not get overloaded?

Delayer

How can I delay the sending of a message?

Load Balancer

How can I balance load across a number of endpoints?

 

Hystrix

To use Hystrix Circuit Breaker when calling an external service.

 

Service Call

To call a remote service in a distributed system where the service is looked up from a service registry of some sorts.

Multicast

How can I route a message to a number of endpoints at the same time?

Loop

How can I repeat processing a message in a loop?

Message Transformation

Content Enricher

How do we communicate with another system if the message originator does not have all the required data items available?

Content Filter

How do you simplify dealing with a large message, when you are interested only in a few data items?

Claim Check

How can we reduce the data volume of message sent across the system without sacrificing information content?

Normalizer

How do you process messages that are semantically equivalent, but arrive in a different format?

Sort

How can I sort the body of a message?

 

Script

How do I execute a script which may not change the message?

Validate

How can I validate a message?

Messaging Endpoints

Messaging Mapper

How do you move data between domain objects and the messaging infrastructure while keeping the two independent of each other?

Event Driven Consumer

How can an application automatically consume messages as they become available?

Polling Consumer

How can an application consume a message when the application is ready?

Competing Consumers

How can a messaging client process multiple messages concurrently?

Message Dispatcher

How can multiple consumers on a single channel coordinate their message processing?

Selective Consumer

How can a message consumer select which messages it wishes to receive?

Durable Subscriber

How can a subscriber avoid missing messages while it's not listening for them?

Idempotent Consumer

How can a message receiver deal with duplicate messages?

Transactional Client

How can a client control its transactions with the messaging system?

Messaging Gateway

How do you encapsulate access to the messaging system from the rest of the application?

Service Activator

How can an application design a service to be invoked both via various messaging technologies and via non-messaging techniques?

System Management

ControlBus

How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area?

Detour

How can you route a message through intermediate steps to perform validation, testing or debugging functions?

Wire Tap

How do you inspect messages that travel on a point-to-point channel?

Message History

How can we effectively analyze and debug the flow of messages in a loosely coupled system?

Log

How can I log processing a message?

For a full breakdown of each pattern see the Book Pattern Appendix

Error rendering macro 'include'

org.owasp.validator.html.ScanException: java.util.EmptyStackException

Tutorials

There now follows the documentation on camel tutorials

We have a number of tutorials as listed below. The tutorials often comes with source code which is either available in the Camel Download or attached to the wiki page.

Notice

These tutorials listed below, is hosted at Apache. We offer the Articles page where we have a link collection for 3rd party Camel material, such as tutorials, blog posts, published articles, videos, pod casts, presentations, and so forth.

If you have written a Camel related article, then we are happy to provide a link to it. You can contact the Camel Team, for example using the Mailing Lists, (or post a tweet with the word Apache Camel).

  • Report Incident - This tutorial introduces Camel steadily and is based on a real life integration problem
    This is a very long tutorial beginning from the start; its for entry level to Camel. Its based on a real life integration, showing how Camel can be introduced in an existing solution. We do this in baby steps. The tutorial is currently work in progress, so check it out from time to time. The tutorial explains some of the inner building blocks Camel uses under the covers. This is good knowledge to have when you start using Camel on a higher abstract level where it can do wonders in a few lines of routing DSL.
  • Tutorial on Camel 1.4 for Integration
    Another real-life scenario. The company sells widgets, with a somewhat unique business process (their customers periodically report what they've purchased in order to get billed). However every customer uses a different data format and protocol. This tutorial goes through the process of integrating (and testing!) several customers and their electronic reporting of the widgets they've bought, along with the company's response.
  • Tutorial how to build a Service Oriented Architecture using Camel with OSGI - Updated 20/11/2009
    The tutorial has been designed in two parts. The first part introduces basic concept to create a simple SOA solution using Camel and OSGI and deploy it in a OSGI Server like Apache Felix Karaf and Spring DM Server while the second extends the ReportIncident tutorial part 4 to show How we can separate the different layers (domain, service, ...) of an application and deploy them in separate bundles. The Web Application has also be modified in order to communicate to the OSGI bundles.
  • Several of the vendors on the Commercial Camel Offerings page also offer various tutorials, webinars, examples, etc.... that may be useful.
  • Examples
    While not actual tutorials you might find working through the source of the various Examples useful.

Tutorial on Spring Remoting with JMS

 

Thanks

This tutorial was kindly donated to Apache Camel by Martin Gilday.

Preface

This tutorial aims to guide the reader through the stages of creating a project which uses Camel to facilitate the routing of messages from a JMS queue to a Spring service. The route works in a synchronous fashion returning a response to the client.

Prerequisites

This tutorial uses Maven to setup the Camel project and for dependencies for artifacts.

Distribution

This sample is distributed with the Camel distribution as examples/camel-example-spring-jms.

About

This tutorial is a simple example that demonstrates more the fact how well Camel is seamless integrated with Spring to leverage the best of both worlds. This sample is client server solution using JMS messaging as the transport. The sample has two flavors of servers and also for clients demonstrating different techniques for easy communication.

The Server is a JMS message broker that routes incoming messages to a business service that does computations on the received message and returns a response.
The EIP patterns used in this sample are:

Pattern

Description

Message Channel

We need a channel so the Clients can communicate with the server.

Message

The information is exchanged using the Camel Message interface.

Message Translator

This is where Camel shines as the message exchange between the Server and the Clients are text based strings with numbers. However our business service uses int for numbers. So Camel can do the message translation automatically.

Message Endpoint

It should be easy to send messages to the Server from the the clients. This is achieved with Camel's powerful Endpoint pattern that even can be more powerful combined with Spring remoting. The tutorial has clients using each kind of technique for this.

Point to Point Channel

The client and server exchange data using point to point using a JMS queue.

Event Driven Consumer

The JMS broker is event driven and is invoked when the client sends a message to the server.

We use the following Camel components:

Component

Description

ActiveMQ

We use Apache ActiveMQ as the JMS broker on the Server side

Bean

We use the bean binding to easily route the messages to our business service. This is a very powerful component in Camel.

File

In the AOP enabled Server we store audit trails as files.

JMS

Used for the JMS messaging

Create the Camel Project

For the purposes of the tutorial a single Maven project will be used for both the client and server. Ideally you would break your application down into the appropriate components.

mvn archetype:generate -DgroupId=org.example -DartifactId=CamelWithJmsAndSpring

Update the POM with Dependencies

First we need to have dependencies for the core Camel jars, spring, jms components, and finally ActiveMQ as the message broker.{snippet:id=e1|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/pom.xml}As we use spring xml configuration for the ActiveMQ JMS broker we need this dependency:{snippet:id=e2|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/pom.xml}

Writing the Server

Create the Spring Service

For this example the Spring service (our business service) on the server will be a simple multiplier which trebles in the received value.{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/server/Multiplier.java}And the implementation of this service is:{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/server/Treble.java}Notice that this class has been annotated with the @Service spring annotation. This ensures that this class is registered as a bean in the registry with the given name multiplier.

Define the Camel Routes

{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/server/ServerRoutes.java}This defines a Camel route from the JMS queue named numbers to the Spring bean named multiplier. Camel will create a consumer to the JMS queue which forwards all received messages onto the the Spring bean, using the method named multiply.

Configure Spring

The Spring config file is placed under META-INF/spring as this is the default location used by the Camel Maven Plugin, which we will later use to run our server.
First we need to do the standard scheme declarations in the top. In the camel-server.xml we are using spring beans as the default bean: namespace and springs context:. For configuring ActiveMQ we use broker: and for Camel we of course have camel:. Notice that we don't use version numbers for the camel-spring schema. At runtime the schema is resolved in the Camel bundle. If we use a specific version number such as 1.4 then its IDE friendly as it would be able to import it and provide smart completion etc. See Xml Reference for further details.{snippet:id=e1|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/META-INF/spring/camel-server.xml}We use Spring annotations for doing IoC dependencies and its component-scan features comes to the rescue as it scans for spring annotations in the given package name:{snippet:id=e2|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/META-INF/spring/camel-server.xml}Camel will of course not be less than Spring in this regard so it supports a similar feature for scanning of Routes. This is configured as shown below.
Notice that we also have enabled the JMXAgent so we will be able to introspect the Camel Server with a JMX Console.{snippet:id=e3|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/META-INF/spring/camel-server.xml}The ActiveMQ JMS broker is also configured in this xml file. We set it up to listen on TCP port 61610.{snippet:id=e4|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/META-INF/spring/camel-server.xml}As this examples uses JMS then Camel needs a JMS component that is connected with the ActiveMQ broker. This is configured as shown below:{snippet:id=e5|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/META-INF/spring/camel-server.xml}Notice: The JMS component is configured in standard Spring beans, but the gem is that the bean id can be referenced from Camel routes - meaning we can do routing using the JMS Component by just using jms: prefix in the route URI. What happens is that Camel will find in the Spring Registry for a bean with the id="jms". Since the bean id can have arbitrary name you could have named it id="jmsbroker" and then referenced to it in the routing as from="jmsbroker:queue:numbers).to("multiplier");
We use the vm protocol to connect to the ActiveMQ server as its embedded in this application.

component-scan

Defines the package to be scanned for Spring stereotype annotations, in this case, to load the "multiplier" bean

camel-context

Defines the package to be scanned for Camel routes. Will find the ServerRoutes class and create the routes contained within it

jms bean

Creates the Camel JMS component

Run the Server

The Server is started using the org.apache.camel.spring.Main class that can start camel-spring application out-of-the-box. The Server can be started in several flavors:

  • as a standard java main application - just start the org.apache.camel.spring.Main class
  • using maven jave:exec
  • using camel:run

In this sample as there are two servers (with and without AOP) we have prepared some profiles in maven to start the Server of your choice.
The server is started with:
mvn compile exec:java -PCamelServer

Writing The Clients

This sample has three clients demonstrating different Camel techniques for communication

  • CamelClient using the ProducerTemplate for Spring template style coding
  • CamelRemoting using Spring Remoting
  • CamelEndpoint using the Message Endpoint EIP pattern using a neutral Camel API

Client Using The ProducerTemplate

We will initially create a client by directly using ProducerTemplate. We will later create a client which uses Spring remoting to hide the fact that messaging is being used.{snippet:id=e1|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/camel-client.xml}{snippet:id=e2|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/camel-client.xml}{snippet:id=e3|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/camel-client.xml}The client will not use the Camel Maven Plugin so the Spring XML has been placed in src/main/resources to not conflict with the server configs.

camelContext

The Camel context is defined but does not contain any routes

template

The ProducerTemplate is used to place messages onto the JMS queue

jms bean

This initialises the Camel JMS component, allowing us to place messages onto the queue

And the CamelClient source code:{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/client/CamelClient.java}The ProducerTemplate is retrieved from a Spring ApplicationContext and used to manually place a message on the "numbers" JMS queue. The requestBody method will use the exchange pattern InOut, which states that the call should be synchronous, and that the caller expects a response.

Before running the client be sure that both the ActiveMQ broker and the CamelServer are running.

Client Using Spring Remoting

Spring Remoting "eases the development of remote-enabled services". It does this by allowing you to invoke remote services through your regular Java interface, masking that a remote service is being called.{snippet:id=e1|lang=xml|url=camel/trunk/examples/camel-example-spring-jms/src/main/resources/camel-client-remoting.xml}The snippet above only illustrates the different and how Camel easily can setup and use Spring Remoting in one line configurations.

The proxy will create a proxy service bean for you to use to make the remote invocations. The serviceInterface property details which Java interface is to be implemented by the proxy. The serviceUrl defines where messages sent to this proxy bean will be directed. Here we define the JMS endpoint with the "numbers" queue we used when working with Camel template directly. The value of the id property is the name that will be the given to the bean when it is exposed through the Spring ApplicationContext. We will use this name to retrieve the service in our client. I have named the bean multiplierProxy simply to highlight that it is not the same multiplier bean as is being used by CamelServer. They are in completely independent contexts and have no knowledge of each other. As you are trying to mask the fact that remoting is being used in a real application you would generally not include proxy in the name.

And the Java client source code:{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/client/CamelClientRemoting.java}Again, the client is similar to the original client, but with some important differences.

  1. The Spring context is created with the new camel-client-remoting.xml
  2. We retrieve the proxy bean instead of a ProducerTemplate. In a non-trivial example you would have the bean injected as in the standard Spring manner.
  3. The multiply method is then called directly. In the client we are now working to an interface. There is no mention of Camel or JMS inside our Java code.

Client Using Message Endpoint EIP Pattern

This client uses the Message Endpoint EIP pattern to hide the complexity to communicate to the Server. The Client uses the same simple API to get hold of the endpoint, create an exchange that holds the message, set the payload and create a producer that does the send and receive. All done using the same neutral Camel API for all the components in Camel. So if the communication was socket TCP based you just get hold of a different endpoint and all the java code stays the same. That is really powerful.

Okay enough talk, show me the code!{snippet:id=e1|lang=java|url=camel/trunk/examples/camel-example-spring-jms/src/main/java/org/apache/camel/example/client/CamelClientEndpoint.java}Switching to a different component is just a matter of using the correct endpoint. So if we had defined a TCP endpoint as: "mina:tcp://localhost:61610" then its just a matter of getting hold of this endpoint instead of the JMS and all the rest of the java code is exactly the same.

Run the Clients

The Clients is started using their main class respectively.

  • as a standard java main application - just start their main class
  • using maven jave:exec

In this sample we start the clients using maven:
mvn compile exec:java -PCamelClient
mvn compile exec:java -PCamelClientRemoting
mvn compile exec:java -PCamelClientEndpoint

Also see the Maven pom.xml file how the profiles for the clients is defined.

Using the Camel Maven Plugin

The Camel Maven Plugin allows you to run your Camel routes directly from Maven. This negates the need to create a host application, as we did with Camel server, simply to start up the container. This can be very useful during development to get Camel routes running quickly.

pom.xml<build> <plugins> <plugin> <groupId>org.apache.camel</groupId> <artifactId>camel-maven-plugin</artifactId> </plugin> </plugins> </build>

All that is required is a new plugin definition in your Maven POM. As we have already placed our Camel config in the default location (camel-server.xml has been placed in META-INF/spring/) we do not need to tell the plugin where the route definitions are located. Simply run mvn camel:run.

Using Camel JMX

Camel has extensive support for JMX and allows us to inspect the Camel Server at runtime. As we have enabled the JMXAgent in our tutorial we can fire up the jconsole and connect to the following service URI: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi/camel. Notice that Camel will log at INFO level the JMX Connector URI:

... DefaultInstrumentationAgent INFO JMX connector thread started on service:jmx:rmi:///jndi/rmi://claus-acer:1099/jmxrmi/camel ...

In the screenshot below we can see the route and its performance metrics:

See Also

Tutorial - camel-example-reportincident

Introduction

Creating this tutorial was inspired by a real life use-case I discussed over the phone with a colleague. He was working at a client whom uses a heavy-weight integration platform from a very large vendor. He was in talks with developer shops to implement a new integration on this platform. His trouble was the shop tripled the price when they realized the platform of choice. So I was wondering how we could do this integration with Camel. Can it be done, without tripling the cost (wink).

This tutorial is written during the development of the integration. I have decided to start off with a sample that isn't Camel's but standard Java and then plugin Camel as we goes. Just as when people needed to learn Spring you could consume it piece by piece, the same goes with Camel.

The target reader is person whom hasn't experience or just started using Camel.

Motivation for this tutorial

I wrote this tutorial motivated as Camel lacked an example application that was based on the web application deployment model. The entire world hasn't moved to pure OSGi deployments yet.

The full source code for this tutorial as complete is part of the Apache Camel distribution in the examples/camel-example-reportincident directory

The use-case

The goal is to allow staff to report incidents into a central administration. For that they use client software where they report the incident and submit it to the central administration. As this is an integration in a transition phase the administration should get these incidents by email whereas they are manually added to the database. The client software should gather the incident and submit the information to the integration platform that in term will transform the report into an email and send it to the central administrator for manual processing.

The figure below illustrates this process. The end users reports the incidents using the client applications. The incident is sent to the central integration platform as webservice. The integration platform will process the incident and send an OK acknowledgment back to the client. Then the integration will transform the message to an email and send it to the administration mail server. The users in the administration will receive the emails and take it from there.

In EIP patterns

We distill the use case as EIP patterns:

Parts

This tutorial is divided into sections and parts:

Section A: Existing Solution, how to slowly use Camel

Part 1 - This first part explain how to setup the project and get a webservice exposed using Apache CXF. In fact we don't touch Camel yet.

Part 2 - Now we are ready to introduce Camel piece by piece (without using Spring or any XML configuration file) and create the full feature integration. This part will introduce different Camel's concepts and How we can build our solution using them like :

  • CamelContext
  • Endpoint, Exchange & Producer
  • Components : Log, File

Part 3 - Continued from part 2 where we implement that last part of the solution with the event driven consumer and how to send the email through the Mail component.

Section B: The Camel Solution

Part 4 - We now turn into the path of Camel where it excels - the routing.
Part 5 - Is about how embed Camel with Spring and using CXF endpoints directly in Camel
Part 6 - Showing a alternative solution primarily using XML instead of Java code

Using Axis 2

See this blog entry by Sagara demonstrating how to use Apache Axis 2 instead of Apache CXF as the web service framework.

Part 1

Prerequisites

This tutorial uses the following frameworks:

  • Maven 3.0.4
  • Apache Camel 2.10.0
  • Apache CXF 2.6.1
  • Spring 3.0.7

Note: The sample project can be downloaded, see the resources section.

Initial Project Setup

We want the integration to be a standard .war application that can be deployed in any web container such as Tomcat, Jetty or even heavy weight application servers such as WebLogic or WebSphere. There fore we start off with the standard Maven webapp project that is created with the following long archetype command:

mvn archetype:create -DgroupId=org.apache.camel -DartifactId=camel-example-reportincident -DarchetypeArtifactId=maven-archetype-webapp

Notice that the groupId etc. doens't have to be org.apache.camel it can be com.mycompany.whatever. But I have used these package names as the example is an official part of the Camel distribution.

Then we have the basic maven folder layout. We start out with the webservice part where we want to use Apache CXF for the webservice stuff. So we add this to the pom.xml

    <properties>
        <cxf-version>2.6.1</cxf-version>
    </properties>

    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-core</artifactId>
        <version>${cxf-version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-frontend-jaxws</artifactId>
        <version>${cxf-version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-transports-http</artifactId>
        <version>${cxf-version}</version>
    </dependency>

Developing the WebService

As we want to develop webservice with the contract first approach we create our .wsdl file. As this is a example we have simplified the model of the incident to only include 8 fields. In real life the model would be a bit more complex, but not to much.

We put the wsdl file in the folder src/main/webapp/WEB-INF/wsdl and name the file report_incident.wsdl.

<?xml version="1.0" encoding="ISO-8859-1"?>
<wsdl:definitions xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
	xmlns:tns="http://reportincident.example.camel.apache.org"
	xmlns:xs="http://www.w3.org/2001/XMLSchema"
	xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
	xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
	targetNamespace="http://reportincident.example.camel.apache.org">

	<!-- Type definitions for input- and output parameters for webservice -->
	<wsdl:types>
	<xs:schema targetNamespace="http://reportincident.example.camel.apache.org">
			<xs:element name="inputReportIncident">
				<xs:complexType>
					<xs:sequence>
						<xs:element type="xs:string"  name="incidentId"/>
						<xs:element type="xs:string"  name="incidentDate"/>
						<xs:element type="xs:string"  name="givenName"/>
						<xs:element type="xs:string"  name="familyName"/>
						<xs:element type="xs:string"  name="summary"/>
						<xs:element type="xs:string"  name="details"/>
						<xs:element type="xs:string"  name="email"/>
						<xs:element type="xs:string"  name="phone"/>
					</xs:sequence>
				</xs:complexType>
			</xs:element>
			<xs:element name="outputReportIncident">
				<xs:complexType>
					<xs:sequence>
						<xs:element type="xs:string" name="code"/>
					</xs:sequence>
				</xs:complexType>
			</xs:element>
		</xs:schema>
	</wsdl:types>

	<!-- Message definitions for input and output -->
	<wsdl:message name="inputReportIncident">
		<wsdl:part name="parameters" element="tns:inputReportIncident"/>
	</wsdl:message>
	<wsdl:message name="outputReportIncident">
		<wsdl:part name="parameters" element="tns:outputReportIncident"/>
	</wsdl:message>

	<!-- Port (interface) definitions -->
	<wsdl:portType name="ReportIncidentEndpoint">
		<wsdl:operation name="ReportIncident">
			<wsdl:input message="tns:inputReportIncident"/>
			<wsdl:output message="tns:outputReportIncident"/>
		</wsdl:operation>
	</wsdl:portType>

	<!-- Port bindings to transports and encoding - HTTP, document literal encoding is used -->
	<wsdl:binding name="ReportIncidentBinding" type="tns:ReportIncidentEndpoint">
		<soap:binding transport="http://schemas.xmlsoap.org/soap/http"/>
		<wsdl:operation name="ReportIncident">
			<soap:operation
				soapAction="http://reportincident.example.camel.apache.org/ReportIncident"
				style="document"/>
			<wsdl:input>
				<soap:body parts="parameters" use="literal"/>
			</wsdl:input>
			<wsdl:output>
				<soap:body parts="parameters" use="literal"/>
			</wsdl:output>
		</wsdl:operation>
	</wsdl:binding>

	<!-- Service definition -->
	<wsdl:service name="ReportIncidentService">
		<wsdl:port name="ReportIncidentPort" binding="tns:ReportIncidentBinding">
			<soap:address location="http://reportincident.example.camel.apache.org"/>
		</wsdl:port>
	</wsdl:service>

</wsdl:definitions>

CXF wsdl2java

Then we integration the CXF wsdl2java generator in the pom.xml so we have CXF generate the needed POJO classes for our webservice contract.
However at first we must configure maven to live in the modern world of Java 1.6 so we must add this to the pom.xml

			<!-- to compile with 1.6 -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>1.6</source>
					<target>1.6</target>
				</configuration>
			</plugin>

And then we can add the CXF wsdl2java code generator that will hook into the compile goal so its automatic run all the time:

			<!-- CXF wsdl2java generator, will plugin to the compile goal -->
			<plugin>
				<groupId>org.apache.cxf</groupId>
				<artifactId>cxf-codegen-plugin</artifactId>
				<version>${cxf-version}</version>
				<executions>
					<execution>
						<id>generate-sources</id>
						<phase>generate-sources</phase>
						<configuration>
							<sourceRoot>${basedir}/target/generated/src/main/java</sourceRoot>
							<wsdlOptions>
								<wsdlOption>
									<wsdl>${basedir}/src/main/webapp/WEB-INF/wsdl/report_incident.wsdl</wsdl>
								</wsdlOption>
							</wsdlOptions>
						</configuration>
						<goals>
							<goal>wsdl2java</goal>
						</goals>
					</execution>
				</executions>
			</plugin>

You are now setup and should be able to compile the project. So running the mvn compile should run the CXF wsdl2java and generate the source code in the folder &{basedir}/target/generated/src/main/java that we specified in the pom.xml above. Since its in the target/generated/src/main/java maven will pick it up and include it in the build process.

Configuration of the web.xml

Next up is to configure the web.xml to be ready to use CXF so we can expose the webservice.
As Spring is the center of the universe, or at least is a very important framework in today's Java land we start with the listener that kick-starts Spring. This is the usual piece of code:

	<!-- the listener that kick-starts Spring -->
	<listener>
		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
	</listener>

And then we have the CXF part where we define the CXF servlet and its URI mappings to which we have chosen that all our webservices should be in the path /webservices/

	<!-- CXF servlet -->
	<servlet>
		<servlet-name>CXFServlet</servlet-name>
		<servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>

	<!-- all our webservices are mapped under this URI pattern -->
	<servlet-mapping>
		<servlet-name>CXFServlet</servlet-name>
		<url-pattern>/webservices/*</url-pattern>
	</servlet-mapping>

Then the last piece of the puzzle is to configure CXF, this is done in a spring XML that we link to fron the web.xml by the standard Spring contextConfigLocation property in the web.xml

	<!-- location of spring xml files -->
	<context-param>
		<param-name>contextConfigLocation</param-name>
		<param-value>classpath:cxf-config.xml</param-value>
	</context-param>

We have named our CXF configuration file cxf-config.xml and its located in the root of the classpath. In Maven land that is we can have the cxf-config.xml file in the src/main/resources folder. We could also have the file located in the WEB-INF folder for instance <param-value>/WEB-INF/cxf-config.xml</param-value>.

Getting rid of the old jsp world

The maven archetype that created the basic folder structure also created a sample .jsp file index.jsp. This file src/main/webapp/index.jsp should be deleted.

Configuration of CXF

The cxf-config.xml is as follows:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:jaxws="http://cxf.apache.org/jaxws"
       xsi:schemaLocation="
            http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">

    <import resource="classpath:META-INF/cxf/cxf.xml"/>
    <import resource="classpath:META-INF/cxf/cxf-extension-soap.xml"/>
    <import resource="classpath:META-INF/cxf/cxf-servlet.xml"/>

    <!-- implementation of the webservice -->
    <bean id="reportIncidentEndpoint" class="org.apache.camel.example.reportincident.ReportIncidentEndpointImpl"/>

    <!-- export the webservice using jaxws -->
    <jaxws:endpoint id="reportIncident"
                    implementor="#reportIncidentEndpoint"
                    address="/incident"
                    wsdlLocation="/WEB-INF/wsdl/report_incident.wsdl"
                    endpointName="s:ReportIncidentPort"
                    serviceName="s:ReportIncidentService"
                    xmlns:s="http://reportincident.example.camel.apache.org"/>

</beans>

The configuration is standard CXF and is documented at the Apache CXF website.

The 3 import elements is needed by CXF and they must be in the file.

Noticed that we have a spring bean reportIncidentEndpoint that is the implementation of the webservice endpoint we let CXF expose.
Its linked from the jaxws element with the implementator attribute as we use the # mark to identify its a reference to a spring bean. We could have stated the classname directly as implementor="org.apache.camel.example.reportincident.ReportIncidentEndpoint" but then we lose the ability to let the ReportIncidentEndpoint be configured by spring.
The address attribute defines the relative part of the URL of the exposed webservice. wsdlLocation is an optional parameter but for persons like me that likes contract-first we want to expose our own .wsdl contracts and not the auto generated by the frameworks, so with this attribute we can link to the real .wsdl file. The last stuff is needed by CXF as you could have several services so it needs to know which this one is. Configuring these is quite easy as all the information is in the wsdl already.

Implementing the ReportIncidentEndpoint

Phew after all these meta files its time for some java code so we should code the implementor of the webservice. So we fire up mvn compile to let CXF generate the POJO classes for our webservice and we are ready to fire up a Java editor.

You can use mvn idea:idea or mvn eclipse:eclipse to create project files for these editors so you can load the project. However IDEA has been smarter lately and can load a pom.xml directly.

As we want to quickly see our webservice we implement just a quick and dirty as it can get. At first beware that since its jaxws and Java 1.5 we get annotations for the money, but they reside on the interface so we can remove them from our implementations so its a nice plain POJO again:

package org.apache.camel.example.reportincident;

/**
 * The webservice we have implemented.
 */
public class ReportIncidentEndpointImpl implements ReportIncidentEndpoint {

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        System.out.println("Hello ReportIncidentEndpointImpl is called from " + parameters.getGivenName());

        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

}

We just output the person that invokes this webservice and returns a OK response. This class should be in the maven source root folder src/main/java under the package name org.apache.camel.example.reportincident. Beware that the maven archetype tool didn't create the src/main/java folder, so you should create it manually.

To test if we are home free we run mvn clean compile.

Running our webservice

Now that the code compiles we would like to run it inside a web container, for this purpose we make use of Jetty which we will bootstrap using it's plugin org.mortbay.jetty:maven-jetty-plugin:

       <build>
           <plugins>
               ...
               <!-- so we can run mvn jetty:run -->
               <plugin>
                   <groupId>org.mortbay.jetty</groupId>
                   <artifactId>maven-jetty-plugin</artifactId>
                   <version>${jetty-version}</version>
               </plugin>

Notice: We make use of the Jetty version being defined inside the Camel's Parent POM.

So to see if everything is in order we fire up jetty with mvn jetty:run and if everything is okay you should be able to access http://localhost:8080.
Jetty is smart that it will list the correct URI on the page to our web application, so just click on the link. This is smart as you don't have to remember the exact web context URI for your application - just fire up the default page and Jetty will help you.

So where is the damn webservice then? Well as we did configure the web.xml to instruct the CXF servlet to accept the pattern /webservices/* we should hit this URL to get the attention of CXF: http://localhost:8080/camel-example-reportincident/webservices.

 

Hitting the webservice

Now we have the webservice running in a standard .war application in a standard web container such as Jetty we would like to invoke the webservice and see if we get our code executed. Unfortunately this isn't the easiest task in the world - its not so easy as a REST URL, so we need tools for this. So we fire up our trusty webservice tool SoapUI and let it be the one to fire the webservice request and see the response.

Using SoapUI we sent a request to our webservice and we got the expected OK response and the console outputs the System.out so we are ready to code.

 

Remote Debugging

Okay a little sidestep but wouldn't it be cool to be able to debug your code when its fired up under Jetty? As Jetty is started from maven, we need to instruct maven to use debug mode.
Se we set the MAVEN_OPTS environment to start in debug mode and listen on port 5005.

MAVEN_OPTS=-Xmx512m -XX:MaxPermSize=128m -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

Then you need to restart Jetty so its stopped with ctrl + c. Remember to start a new shell to pickup the new environment settings. And start jetty again.

Then we can from our IDE attach a remote debugger and debug as we want.
First we configure IDEA to attach to a remote debugger on port 5005:

 

Then we set a breakpoint in our code ReportIncidentEndpoint and hit the SoapUI once again and we are breaked at the breakpoint where we can inspect the parameters:

 

Adding a unit test

Oh so much hard work just to hit a webservice, why can't we just use an unit test to invoke our webservice? Yes of course we can do this, and that's the next step.
First we create the folder structure src/test/java and src/test/resources. We then create the unit test in the src/test/java folder.

package org.apache.camel.example.reportincident;

import junit.framework.TestCase;

/**
 * Plain JUnit test of our webservice.
 */
public class ReportIncidentEndpointTest extends TestCase {

}

Here we have a plain old JUnit class. As we want to test webservices we need to start and expose our webservice in the unit test before we can test it. And JAXWS has pretty decent methods to help us here, the code is simple as:

    import javax.xml.ws.Endpoint;
    ...

    private static String ADDRESS = "http://localhost:9090/unittest";

    protected void startServer() throws Exception {
        // We need to start a server that exposes or webservice during the unit testing
        // We use jaxws to do this pretty simple
        ReportIncidentEndpointImpl server = new ReportIncidentEndpointImpl();
        Endpoint.publish(ADDRESS, server);
    }

The Endpoint class is the javax.xml.ws.Endpoint that under the covers looks for a provider and in our case its CXF - so its CXF that does the heavy lifting of exposing out webservice on the given URL address. Since our class ReportIncidentEndpointImpl implements the interface ReportIncidentEndpoint that is decorated with all the jaxws annotations it got all the information it need to expose the webservice. Below is the CXF wsdl2java generated interface:


/*
 * 
 */

package org.apache.camel.example.reportincident;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;
import javax.jws.soap.SOAPBinding.ParameterStyle;
import javax.xml.bind.annotation.XmlSeeAlso;

/**
 * This class was generated by Apache CXF 2.1.1
 * Wed Jul 16 12:40:31 CEST 2008
 * Generated source version: 2.1.1
 * 
 */
 
 /*
  * 
  */


@WebService(targetNamespace = "http://reportincident.example.camel.apache.org", name = "ReportIncidentEndpoint")
@XmlSeeAlso({ObjectFactory.class})
@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)

public interface ReportIncidentEndpoint {

/*
 * 
 */

    @SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)
    @WebResult(name = "outputReportIncident", targetNamespace = "http://reportincident.example.camel.apache.org", partName = "parameters")
    @WebMethod(operationName = "ReportIncident", action = "http://reportincident.example.camel.apache.org/ReportIncident")
    public OutputReportIncident reportIncident(
        @WebParam(partName = "parameters", name = "inputReportIncident", targetNamespace = "http://reportincident.example.camel.apache.org")
        InputReportIncident parameters
    );
}

Next up is to create a webservice client so we can invoke our webservice. For this we actually use the CXF framework directly as its a bit more easier to create a client using this framework than using the JAXWS style. We could have done the same for the server part, and you should do this if you need more power and access more advanced features.

    import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;
    ...
    
    protected ReportIncidentEndpoint createCXFClient() {
        // we use CXF to create a client for us as its easier than JAXWS and works
        JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();
        factory.setServiceClass(ReportIncidentEndpoint.class);
        factory.setAddress(ADDRESS);
        return (ReportIncidentEndpoint) factory.create();
    }

So now we are ready for creating a unit test. We have the server and the client. So we just create a plain simple unit test method as the usual junit style:

    public void testRendportIncident() throws Exception {
        startServer();

        ReportIncidentEndpoint client = createCXFClient();

        InputReportIncident input = new InputReportIncident();
        input.setIncidentId("123");
        input.setIncidentDate("2008-07-16");
        input.setGivenName("Claus");
        input.setFamilyName("Ibsen");
        input.setSummary("bla bla");
        input.setDetails("more bla bla");
        input.setEmail("davsclaus@apache.org");
        input.setPhone("+45 2962 7576");

        OutputReportIncident out = client.reportIncident(input);
        assertEquals("Response code is wrong", "OK", out.getCode());
    }

Now we are nearly there. But if you run the unit test with mvn test then it will fail. Why!!! Well its because that CXF needs is missing some dependencies during unit testing. In fact it needs the web container, so we need to add this to our pom.xml.

    <!-- cxf web container for unit testing -->
    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-transports-http-jetty</artifactId>
        <version>${cxf-version}</version>
        <scope>test</scope>
    </dependency>

Well what is that, CXF also uses Jetty for unit test - well its just shows how agile, embedable and popular Jetty is.

So lets run our junit test with, and it reports:

mvn test
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESSFUL

Yep thats it for now. We have a basic project setup.

End of part 1

Thanks for being patient and reading all this more or less standard Maven, Spring, JAXWS and Apache CXF stuff. Its stuff that is well covered on the net, but I wanted a full fledged tutorial on a maven project setup that is web service ready with Apache CXF. We will use this as a base for the next part where we demonstrate how Camel can be digested slowly and piece by piece just as it was back in the times when was introduced and was learning the Spring framework that we take for granted today.

#Resources

Links

Part 2

Adding Camel

In this part we will introduce Camel so we start by adding Camel to our pom.xml:

       <properties>
            ...
            <camel-version>1.4.0</camel-version>
        </properties>

        <!-- camel -->
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-core</artifactId>
            <version>${camel-version}</version>
        </dependency>

That's it, only one dependency for now.

Synchronize IDE

If you continue from part 1, remember to update your editor project settings since we have introduce new .jar files. For instance IDEA has a feature to synchronize with Maven projects.

Now we turn towards our webservice endpoint implementation where we want to let Camel have a go at the input we receive. As Camel is very non invasive its basically a .jar file then we can just grap Camel but creating a new instance of DefaultCamelContext that is the hearth of Camel its context.

CamelContext camel = new DefaultCamelContext();

In fact we create a constructor in our webservice and add this code:

    private CamelContext camel;

    public ReportIncidentEndpointImpl() throws Exception {
        // create the camel context that is the "heart" of Camel
        camel = new DefaultCamelContext();

        // add the log component
        camel.addComponent("log", new LogComponent());

        // start Camel
        camel.start();
    }

Logging the "Hello World"

Here at first we want Camel to log the givenName and familyName parameters we receive, so we add the LogComponent with the key log. And we must start Camel before its ready to act.

Component Documentation

The Log and File components is documented as well, just click on the links. Just return to this documentation later when you must use these components for real.

Then we change the code in the method that is invoked by Apache CXF when a webservice request arrives. We get the name and let Camel have a go at it in the new method we create sendToCamel:

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        String name = parameters.getGivenName() + " " + parameters.getFamilyName();

        // let Camel do something with the name
        sendToCamelLog(name);

        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

Next is the Camel code. At first it looks like there are many code lines to do a simple task of logging the name - yes it is. But later you will in fact realize this is one of Camels true power. Its concise API. Hint: The same code can be used for any component in Camel.

    private void sendToCamelLog(String name) {
        try {
            // get the log component
            Component component = camel.getComponent("log");

            // create an endpoint and configure it.
            // Notice the URI parameters this is a common pratice in Camel to configure
            // endpoints based on URI.
            // com.mycompany.part2 = the log category used. Will log at INFO level as default
            Endpoint endpoint = component.createEndpoint("log:com.mycompany.part2");

            // create an Exchange that we want to send to the endpoint
            Exchange exchange = endpoint.createExchange();
            // set the in message payload (=body) with the name parameter
            exchange.getIn().setBody(name);

            // now we want to send the exchange to this endpoint and we then need a producer
            // for this, so we create and start the producer.
            Producer producer = endpoint.createProducer();
            producer.start();
            // process the exchange will send the exchange to the log component, that will process
            // the exchange and yes log the payload
            producer.process(exchange);

            // stop the producer, we want to be nice and cleanup
            producer.stop();




        } catch (Exception e) {
            // we ignore any exceptions and just rethrow as runtime
            throw new RuntimeException(e);

        }
    }

Okay there are code comments in the code block above that should explain what is happening. We run the code by invoking our unit test with maven mvn test, and we should get this log line:

INFO: Exchange[BodyType:String, Body:Claus Ibsen]

Write to file - easy with the same code style

Okay that isn't to impressive, Camel can log (wink) Well I promised that the above code style can be used for any component, so let's store the payload in a file. We do this by adding the file component to the Camel context

        // add the file component
        camel.addComponent("file", new FileComponent());

And then we let camel write the payload to the file after we have logged, by creating a new method sendToCamelFile. We want to store the payload in filename with the incident id so we need this parameter also:

        // let Camel do something with the name
        sendToCamelLog(name);
        sendToCamelFile(parameters.getIncidentId(), name);

And then the code that is 99% identical. We have change the URI configuration when we create the endpoint as we pass in configuration parameters to the file component.
And then we need to set the output filename and this is done by adding a special header to the exchange. That's the only difference:

    private void sendToCamelFile(String incidentId, String name) {
        try {
            // get the file component
            Component component = camel.getComponent("file");

            // create an endpoint and configure it.
            // Notice the URI parameters this is a common pratice in Camel to configure
            // endpoints based on URI.
            // file://target instructs the base folder to output the files. We put in the target folder
            // then its actumatically cleaned by mvn clean
            Endpoint endpoint = component.createEndpoint("file://target");

            // create an Exchange that we want to send to the endpoint
            Exchange exchange = endpoint.createExchange();
            // set the in message payload (=body) with the name parameter
            exchange.getIn().setBody(name);

            // now a special header is set to instruct the file component what the output filename
            // should be
            exchange.getIn().setHeader(FileComponent.HEADER_FILE_NAME, "incident-" + incidentId + ".txt");

            // now we want to send the exchange to this endpoint and we then need a producer
            // for this, so we create and start the producer.
            Producer producer = endpoint.createProducer();
            producer.start();
            // process the exchange will send the exchange to the file component, that will process
            // the exchange and yes write the payload to the given filename
            producer.process(exchange);

            // stop the producer, we want to be nice and cleanup
            producer.stop();
        } catch (Exception e) {
            // we ignore any exceptions and just rethrow as runtime
            throw new RuntimeException(e);
        }
    }

After running our unit test again with mvn test we have a output file in the target folder:

D:\demo\part-two>type target\incident-123.txt
Claus Ibsen

Fully java based configuration of endpoints

In the file example above the configuration was URI based. What if you want 100% java setter based style, well this is of course also possible. We just need to cast to the component specific endpoint and then we have all the setters available:

            // create the file endpoint, we cast to FileEndpoint because then we can do
            // 100% java settter based configuration instead of the URI sting based
            // must pass in an empty string, or part of the URI configuration if wanted 
            FileEndpoint endpoint = (FileEndpoint)component.createEndpoint("");
            endpoint.setFile(new File("target/subfolder"));
            endpoint.setAutoCreate(true);

That's it. Now we have used the setters to configure the FileEndpoint that it should store the file in the folder target/subfolder. Of course Camel now stores the file in the subfolder.

D:\demo\part-two>type target\subfolder\incident-123.txt
Claus Ibsen

Lessons learned

Okay I wanted to demonstrate how you can be in 100% control of the configuration and usage of Camel based on plain Java code with no hidden magic or special XML or other configuration files. Just add the camel-core.jar and you are ready to go.

You must have noticed that the code for sending a message to a given endpoint is the same for both the log and file, in fact any Camel endpoint. You as the client shouldn't bother with component specific code such as file stuff for file components, jms stuff for JMS messaging etc. This is what the Message Endpoint EIP pattern is all about and Camel solves this very very nice - a key pattern in Camel.

Reducing code lines

Now that you have been introduced to Camel and one of its masterpiece patterns solved elegantly with the Message Endpoint its time to give productive and show a solution in fewer code lines, in fact we can get it down to 5, 4, 3, 2 .. yes only 1 line of code.

The key is the ProducerTemplate that is a Spring'ish xxxTemplate based producer. Meaning that it has methods to send messages to any Camel endpoints. First of all we need to get hold of such a template and this is done from the CamelContext

    private ProducerTemplate template;

    public ReportIncidentEndpointImpl() throws Exception {
        ...

        // get the ProducerTemplate thst is a Spring'ish xxxTemplate based producer for very
        // easy sending exchanges to Camel.
        template = camel.createProducerTemplate();

        // start Camel
        camel.start();
    }

Now we can use template for sending payloads to any endpoint in Camel. So all the logging gabble can be reduced to:

    template.sendBody("log:com.mycompany.part2.easy", name);

And the same goes for the file, but we must also send the header to instruct what the output filename should be:

    String filename = "easy-incident-" + incidentId + ".txt";
    template.sendBodyAndHeader("file://target/subfolder", name, FileComponent.HEADER_FILE_NAME, filename);

Reducing even more code lines

Well we got the Camel code down to 1-2 lines for sending the message to the component that does all the heavy work of wring the message to a file etc. But we still got 5 lines to initialize Camel.

    camel = new DefaultCamelContext();
    camel.addComponent("log", new LogComponent());
    camel.addComponent("file", new FileComponent());
    template = camel.createProducerTemplate();
    camel.start();

This can also be reduced. All the standard components in Camel is auto discovered on-the-fly so we can remove these code lines and we are down to 3 lines.

Component auto discovery

When an endpoint is requested with a scheme that Camel hasn't seen before it will try to look for it in the classpath. It will do so by looking for special Camel component marker files that reside in the folder META-INF/services/org/apache/camel/component. If there are files in this folder it will read them as the filename is the scheme part of the URL. For instance the log component is defined in this file META-INF/services/org/apache/component/log and its content is:

class=org.apache.camel.component.log.LogComponent

The class property defines the component implementation.

Tip: End-users can create their 3rd party components using the same technique and have them been auto discovered on-the-fly.

Okay back to the 3 code lines:

    camel = new DefaultCamelContext();
    template = camel.createProducerTemplate();
    camel.start();

Later will we see how we can reduce this to ... in fact 0 java code lines. But the 3 lines will do for now.

Message Translation

Okay lets head back to the over goal of the integration. Looking at the EIP diagrams at the introduction page we need to be able to translate the incoming webservice to an email. Doing so we need to create the email body. When doing the message translation we could put up our sleeves and do it manually in pure java with a StringBuilder such as:

    private String createMailBody(InputReportIncident parameters) {
        StringBuilder sb = new StringBuilder();
        sb.append("Incident ").append(parameters.getIncidentId());
        sb.append(" has been reported on the ").append(parameters.getIncidentDate());
        sb.append(" by ").append(parameters.getGivenName());
        sb.append(" ").append(parameters.getFamilyName());
        
        // and the rest of the mail body with more appends to the string builder
        
        return sb.toString();
    }

But as always it is a hardcoded template for the mail body and the code gets kinda ugly if the mail message has to be a bit more advanced. But of course it just works out-of-the-box with just classes already in the JDK.

Lets use a template language instead such as Apache Velocity. As Camel have a component for Velocity integration we will use this component. Looking at the Component List overview we can see that camel-velocity component uses the artifactId camel-velocity so therefore we need to add this to the pom.xml

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-velocity</artifactId>
            <version>${camel-version}</version>
        </dependency>

And now we have a Spring conflict as Apache CXF is dependent on Spring 2.0.8 and camel-velocity is dependent on Spring 2.5.5. To remedy this we could wrestle with the pom.xml with excludes settings in the dependencies or just bring in another dependency camel-spring:

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-spring</artifactId>
            <version>${camel-version}</version>
        </dependency>

In fact camel-spring is such a vital part of Camel that you will end up using it in nearly all situations - we will look into how well Camel is seamless integration with Spring in part 3. For now its just another dependency.

We create the mail body with the Velocity template and create the file src/main/resources/MailBody.vm. The content in the MailBody.vm file is:

Incident $body.incidentId has been reported on the $body.incidentDate by $body.givenName $body.familyName.

The person can be contact by:
- email: $body.email
- phone: $body.phone

Summary: $body.summary

Details:
$body.details

This is an auto generated email. You can not reply.

Letting Camel creating the mail body and storing it as a file is as easy as the following 3 code lines:

    private void generateEmailBodyAndStoreAsFile(InputReportIncident parameters) {
        // generate the mail body using velocity template
        // notice that we just pass in our POJO (= InputReportIncident) that we
        // got from Apache CXF to Velocity.
        Object response = template.sendBody("velocity:MailBody.vm", parameters);
        // Note: the response is a String and can be cast to String if needed

        // store the mail in a file
        String filename = "mail-incident-" + parameters.getIncidentId() + ".txt";
        template.sendBodyAndHeader("file://target/subfolder", response, FileComponent.HEADER_FILE_NAME, filename);
    }

What is impressive is that we can just pass in our POJO object we got from Apache CXF to Velocity and it will be able to generate the mail body with this object in its context. Thus we don't need to prepare anything before we let Velocity loose and generate our mail body. Notice that the template method returns a object with out response. This object contains the mail body as a String object. We can cast to String if needed.

If we run our unit test with mvn test we can in fact see that Camel has produced the file and we can type its content:

D:\demo\part-two>type target\subfolder\mail-incident-123.txt
Incident 123 has been reported on the 2008-07-16 by Claus Ibsen.

The person can be contact by:
- email: davsclaus@apache.org
- phone: +45 2962 7576

Summary: bla bla

Details:
more bla bla

This is an auto generated email. You can not reply.

First part of the solution

What we have seen here is actually what it takes to build the first part of the integration flow. Receiving a request from a webservice, transform it to a mail body and store it to a file, and return an OK response to the webservice. All possible within 10 lines of code. So lets wrap it up here is what it takes:

/**
 * The webservice we have implemented.
 */
public class ReportIncidentEndpointImpl implements ReportIncidentEndpoint {

    private CamelContext camel;
    private ProducerTemplate template;

    public ReportIncidentEndpointImpl() throws Exception {
        // create the camel context that is the "heart" of Camel
        camel = new DefaultCamelContext();

        // get the ProducerTemplate thst is a Spring'ish xxxTemplate based producer for very
        // easy sending exchanges to Camel.
        template = camel.createProducerTemplate();

        // start Camel
        camel.start();
    }

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        // transform the request into a mail body
        Object mailBody = template.sendBody("velocity:MailBody.vm", parameters);

        // store the mail body in a file
        String filename = "mail-incident-" + parameters.getIncidentId() + ".txt";
        template.sendBodyAndHeader("file://target/subfolder", mailBody, FileComponent.HEADER_FILE_NAME, filename);

        // return an OK reply
        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

}

Okay I missed by one, its in fact only 9 lines of java code and 2 fields.

End of part 2

I know this is a bit different introduction to Camel to how you can start using it in your projects just as a plain java .jar framework that isn't invasive at all. I took you through the coding parts that requires 6 - 10 lines to send a message to an endpoint, buts it's important to show the Message Endpoint EIP pattern in action and how its implemented in Camel. Yes of course Camel also has to one liners that you can use, and will use in your projects for sending messages to endpoints. This part has been about good old plain java, nothing fancy with Spring, XML files, auto discovery, OGSi or other new technologies. I wanted to demonstrate the basic building blocks in Camel and how its setup in pure god old fashioned Java. There are plenty of eye catcher examples with one liners that does more than you can imagine - we will come there in the later parts.

Okay part 3 is about building the last pieces of the solution and now it gets interesting since we have to wrestle with the event driven consumer.
Brew a cup of coffee, tug the kids and kiss the wife, for now we will have us some fun with the Camel. See you in part 3.

Links

Part 3

Recap

Lets just recap on the solution we have now:

public class ReportIncidentEndpointImpl implements ReportIncidentEndpoint {

    private CamelContext camel;
    private ProducerTemplate template;

    public ReportIncidentEndpointImpl() throws Exception {
        // create the camel context that is the "heart" of Camel
        camel = new DefaultCamelContext();

        // get the ProducerTemplate thst is a Spring'ish xxxTemplate based producer for very
        // easy sending exchanges to Camel.
        template = camel.createProducerTemplate();

        // start Camel
        camel.start();
    }

    /**
     * This is the last solution displayed that is the most simple
     */
    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        // transform the request into a mail body
        Object mailBody = template.sendBody("velocity:MailBody.vm", parameters);

        // store the mail body in a file
        String filename = "mail-incident-" + parameters.getIncidentId() + ".txt";
        template.sendBodyAndHeader("file://target/subfolder", mailBody, FileComponent.HEADER_FILE_NAME, filename);

        // return an OK reply
        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

}

This completes the first part of the solution: receiving the message using webservice, transform it to a mail body and store it as a text file.
What is missing is the last part that polls the text files and send them as emails. Here is where some fun starts, as this requires usage of the Event Driven Consumer EIP pattern to react when new files arrives. So lets see how we can do this in Camel. There is a saying: Many roads lead to Rome, and that is also true for Camel - there are many ways to do it in Camel.

Adding the Event Driven Consumer

We want to add the consumer to our integration that listen for new files, we do this by creating a private method where the consumer code lives. We must register our consumer in Camel before its started so we need to add, and there fore we call the method addMailSenderConsumer in the constructor below:

    public ReportIncidentEndpointImpl() throws Exception {
        // create the camel context that is the "heart" of Camel
        camel = new DefaultCamelContext();

        // get the ProducerTemplate thst is a Spring'ish xxxTemplate based producer for very
        // easy sending exchanges to Camel.
        template = camel.createProducerTemplate();

        // add the event driven consumer that will listen for mail files and process them
        addMailSendConsumer();

        // start Camel
        camel.start();
    }

The consumer needs to be consuming from an endpoint so we grab the endpoint from Camel we want to consume. It's file://target/subfolder. Don't be fooled this endpoint doesn't have to 100% identical to the producer, i.e. the endpoint we used in the previous part to create and store the files. We could change the URL to include some options, and to make it more clear that it's possible we setup a delay value to 10 seconds, and the first poll starts after 2 seconds. This is done by adding ?consumer.delay=10000&consumer.initialDelay=2000 to the URL.

URL Configuration

The URL configuration in Camel endpoints is just like regular URL we know from the Internet. You use ? and & to set the options.

When we have the endpoint we can create the consumer (just as in part 1 where we created a producer}. Creating the consumer requires a Processor where we implement the java code what should happen when a message arrives. To get the mail body as a String object we can use the getBody method where we can provide the type we want in return.

Camel Type Converter

Why don't we just cast it as we always do in Java? Well the biggest advantage when you provide the type as a parameter you tell Camel what type you want and Camel can automatically convert it for you, using its flexible Type Converter mechanism. This is a great advantage, and you should try to use this instead of regular type casting.

Sending the email is still left to be implemented, we will do this later. And finally we must remember to start the consumer otherwise its not active and won't listen for new files.

    private void addMailSendConsumer() throws Exception {
        // Grab the endpoint where we should consume. Option - the first poll starts after 2 seconds
        Endpoint endpint = camel.getEndpoint("file://target/subfolder?consumer.initialDelay=2000");

        // create the event driven consumer
        // the Processor is the code what should happen when there is an event
        // (think it as the onMessage method)
        Consumer consumer = endpint.createConsumer(new Processor() {
            public void process(Exchange exchange) throws Exception {
                // get the mail body as a String
                String mailBody = exchange.getIn().getBody(String.class);

                // okay now we are read to send it as an email
                System.out.println("Sending email..." + mailBody);
            }
        });

        // star the consumer, it will listen for files
        consumer.start();
    }

Before we test it we need to be aware that our unit test is only catering for the first part of the solution, receiving the message with webservice, transforming it using Velocity and then storing it as a file - it doesn't test the Event Driven Consumer we just added. As we are eager to see it in action, we just do a common trick adding some sleep in our unit test, that gives our Event Driven Consumer time to react and print to System.out. We will later refine the test:

    public void testRendportIncident() throws Exception {
       ...

        OutputReportIncident out = client.reportIncident(input);
        assertEquals("Response code is wrong", "OK", out.getCode());

        // give the event driven consumer time to react
        Thread.sleep(10 * 1000);
    }

We run the test with mvn clean test and have eyes fixed on the console output.
During all the output in the console, we see that our consumer has been triggered, as we want.

2008-07-19 12:09:24,140 [mponent@1f12c4e] DEBUG FileProcessStrategySupport - Locking the file: target\subfolder\mail-incident-123.txt ...
Sending email...Incident 123 has been reported on the 2008-07-16 by Claus Ibsen.

The person can be contact by:
- email: davsclaus@apache.org
- phone: +45 2962 7576

Summary: bla bla

Details:
more bla bla

This is an auto generated email. You can not reply.
2008-07-19 12:09:24,156 [mponent@1f12c4e] DEBUG FileConsumer - Done processing file: target\subfolder\mail-incident-123.txt. Status is: OK

Sending the email

Sending the email requires access to a SMTP mail server, but the implementation code is very simple:

    private void sendEmail(String body) {
        // send the email to your mail server
        String url = "smtp://someone@localhost?password=secret&to=incident@mycompany.com";
        template.sendBodyAndHeader(url, body, "subject", "New incident reported");
    }

And just invoke the method from our consumer:

    // okay now we are read to send it as an email
    System.out.println("Sending email...");
    sendEmail(mailBody);
    System.out.println("Email sent");

Unit testing mail

For unit testing the consumer part we will use a mock mail framework, so we add this to our pom.xml:

        <!-- unit testing mail using mock -->
        <dependency>
            <groupId>org.jvnet.mock-javamail</groupId>
            <artifactId>mock-javamail</artifactId>
            <version>1.7</version>
            <scope>test</scope>
        </dependency>

Then we prepare our integration to run with or without the consumer enabled. We do this to separate the route into the two parts:

  • receive the webservice, transform and save mail file and return OK as repose
  • the consumer that listen for mail files and send them as emails

So we change the constructor code a bit:

    public ReportIncidentEndpointImpl() throws Exception {
        init(true);
    }

    public ReportIncidentEndpointImpl(boolean enableConsumer) throws Exception {
        init(enableConsumer);
    }

    private void init(boolean enableConsumer) throws Exception {
        // create the camel context that is the "heart" of Camel
        camel = new DefaultCamelContext();

        // get the ProducerTemplate thst is a Spring'ish xxxTemplate based producer for very
        // easy sending exchanges to Camel.
        template = camel.createProducerTemplate();

        // add the event driven consumer that will listen for mail files and process them
        if (enableConsumer) {
            addMailSendConsumer();
        }

        // start Camel
        camel.start();
    }

Then remember to change the ReportIncidentEndpointTest to pass in false in the ReportIncidentEndpointImpl constructor.
And as always run mvn clean test to be sure that the latest code changes works.

Adding new unit test

We are now ready to add a new unit test that tests the consumer part so we create a new test class that has the following code structure:

/**
 * Plain JUnit test of our consumer.
 */
public class ReportIncidentConsumerTest extends TestCase {

    private ReportIncidentEndpointImpl endpoint;

    public void testConsumer() throws Exception {
        // we run this unit test with the consumer, hence the true parameter
        endpoint = new ReportIncidentEndpointImpl(true);
   }

}

As we want to test the consumer that it can listen for files, read the file content and send it as an email to our mailbox we will test it by asserting that we receive 1 mail in our mailbox and that the mail is the one we expect. To do so we need to grab the mailbox with the mockmail API. This is done as simple as:

    public void testConsumer() throws Exception {
        // we run this unit test with the consumer, hence the true parameter
        endpoint = new ReportIncidentEndpointImpl(true);

        // get the mailbox
        Mailbox box = Mailbox.get("incident@mycompany.com");
        assertEquals("Should not have mails", 0, box.size());

How do we trigger the consumer? Well by creating a file in the folder it listen for. So we could use plain java.io.File API to create the file, but wait isn't there an smarter solution? ... yes Camel of course. Camel can do amazing stuff in one liner codes with its ProducerTemplate, so we need to get a hold of this baby. We expose this template in our ReportIncidentEndpointImpl but adding this getter:

    protected ProducerTemplate getTemplate() {
        return template;
    }

Then we can use the template to create the file in one code line:

        // drop a file in the folder that the consumer listen
        // here is a trick to reuse Camel! so we get the producer template and just
        // fire a message that will create the file for us
        endpoint.getTemplate().sendBodyAndHeader("file://target/subfolder?append=false", "Hello World",
            FileComponent.HEADER_FILE_NAME, "mail-incident-test.txt");

Then we just need to wait a little for the consumer to kick in and do its work and then we should assert that we got the new mail. Easy as just:

        // let the consumer have time to run
        Thread.sleep(3 * 1000);

        // get the mock mailbox and check if we got mail ;)
        assertEquals("Should have got 1 mail", 1, box.size());
        assertEquals("Subject wrong", "New incident reported", box.get(0).getSubject());
        assertEquals("Mail body wrong", "Hello World", box.get(0).getContent());
    }

The final class for the unit test is:

/**
 * Plain JUnit test of our consumer.
 */
public class ReportIncidentConsumerTest extends TestCase {

    private ReportIncidentEndpointImpl endpoint;

    public void testConsumer() throws Exception {
        // we run this unit test with the consumer, hence the true parameter
        endpoint = new ReportIncidentEndpointImpl(true);

        // get the mailbox
        Mailbox box = Mailbox.get("incident@mycompany.com");
        assertEquals("Should not have mails", 0, box.size());

        // drop a file in the folder that the consumer listen
        // here is a trick to reuse Camel! so we get the producer template and just
        // fire a message that will create the file for us
        endpoint.getTemplate().sendBodyAndHeader("file://target/subfolder?append=false", "Hello World",
            FileComponent.HEADER_FILE_NAME, "mail-incident-test.txt");

        // let the consumer have time to run
        Thread.sleep(3 * 1000);

        // get the mock mailbox and check if we got mail ;)
        assertEquals("Should have got 1 mail", 1, box.size());
        assertEquals("Subject wrong", "New incident reported", box.get(0).getSubject());
        assertEquals("Mail body wrong", "Hello World", box.get(0).getContent());
    }

}

End of part 3

Okay we have reached the end of part 3. For now we have only scratched the surface of what Camel is and what it can do. We have introduced Camel into our integration piece by piece and slowly added more and more along the way. And the most important is: you as the developer never lost control. We hit a sweet spot in the webservice implementation where we could write our java code. Adding Camel to the mix is just to use it as a regular java code, nothing magic. We were in control of the flow, we decided when it was time to translate the input to a mail body, we decided when the content should be written to a file. This is very important to not lose control, that the bigger and heavier frameworks tend to do. No names mentioned, but boy do developers from time to time dislike these elephants. And Camel is no elephant.

I suggest you download the samples from part 1 to 3 and try them out. It is great basic knowledge to have in mind when we look at some of the features where Camel really excel - the routing domain language.

From part 1 to 3 we touched concepts such as::

Links

Part 4

Introduction

This section is about regular Camel. The examples presented here in this section is much more in common of all the examples we have in the Camel documentation.

If you have been reading the previous 3 parts then, this quote applies:

you must unlearn what you have learned
Master Yoda, Star Wars IV

So we start all over again! (wink)

Routing

Camel is particular strong as a light-weight and agile routing and mediation framework. In this part we will introduce the routing concept and how we can introduce this into our solution.
Looking back at the figure from the Introduction page we want to implement this routing. Camel has support for expressing this routing logic using Java as a DSL (Domain Specific Language). In fact Camel also has DSL for XML and Scala. In this part we use the Java DSL as its the most powerful and all developers know Java. Later we will introduce the XML version that is very well integrated with Spring.

Before we jump into it, we want to state that this tutorial is about Developers not loosing control. In my humble experience one of the key fears of developers is that they are forced into a tool/framework where they loose control and/or power, and the possible is now impossible. So in this part we stay clear with this vision and our starting point is as follows:

  • We have generated the webservice source code using the CXF wsdl2java generator and we have our ReportIncidentEndpointImpl.java file where we as a Developer feels home and have the power.

So the starting point is:

/**
 * The webservice we have implemented.
 */
public class ReportIncidentEndpointImpl implements ReportIncidentEndpoint {

    /**
     * This is the last solution displayed that is the most simple
     */
    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        // WE ARE HERE !!!
        return null;
    }

}

Yes we have a simple plain Java class where we have the implementation of the webservice. The cursor is blinking at the WE ARE HERE block and this is where we feel home. More or less any Java Developers have implemented webservices using a stack such as: Apache AXIS, Apache CXF or some other quite popular framework. They all allow the developer to be in control and implement the code logic as plain Java code. Camel of course doesn't enforce this to be any different. Okay the boss told us to implement the solution from the figure in the Introduction page and we are now ready to code.

RouteBuilder

RouteBuilder is the hearth in Camel of the Java DSL routing. This class does all the heavy lifting of supporting EIP verbs for end-users to express the routing. It does take a little while to get settled and used to, but when you have worked with it for a while you will enjoy its power and realize it is in fact a little language inside Java itself. Camel is the only integration framework we are aware of that has Java DSL, all the others are usually only XML based.

As an end-user you usually use the RouteBuilder as of follows:

  • create your own Route class that extends RouteBuilder
  • implement your routing DSL in the configure method

So we create a new class ReportIncidentRoutes and implement the first part of the routing:

import org.apache.camel.builder.RouteBuilder;

public class ReportIncidentRoutes extends RouteBuilder {

    public void configure() throws Exception {
        // direct:start is a internal queue to kick-start the routing in our example
        // we use this as the starting point where you can send messages to direct:start
        from("direct:start")
            // to is the destination we send the message to our velocity endpoint
            // where we transform the mail body
            .to("velocity:MailBody.vm");
    }

}

What to notice here is the configure method. Here is where all the action is. Here we have the Java DSL langauge, that is expressed using the fluent builder syntax that is also known from Hibernate when you build the dynamic queries etc. What you do is that you can stack methods separating with the dot.

In the example above we have a very common routing, that can be distilled from pseudo verbs to actual code with:

  • from A to B
  • From Endpoint A To Endpoint B
  • from("endpointA").to("endpointB")
  • from("direct:start").to("velocity:MailBody.vm");

from("direct:start") is the consumer that is kick-starting our routing flow. It will wait for messages to arrive on the direct queue and then dispatch the message.
to("velocity:MailBody.vm") is the producer that will receive a message and let Velocity generate the mail body response.

So what we have implemented so far with our ReportIncidentRoutes RouteBuilder is this part of the picture:

Adding the RouteBuilder

Now we have our RouteBuilder we need to add/connect it to our CamelContext that is the hearth of Camel. So turning back to our webservice implementation class ReportIncidentEndpointImpl we add this constructor to the code, to create the CamelContext and add the routes from our route builder and finally to start it.

    private CamelContext context;

    public ReportIncidentEndpointImpl() throws Exception {
        // create the context
        context = new DefaultCamelContext();

        // append the routes to the context
        context.addRoutes(new ReportIncidentRoutes());

        // at the end start the camel context
        context.start();
    }

Okay how do you use the routes then? Well its just as before we use a ProducerTemplate to send messages to Endpoints, so we just send to the direct:start endpoint and it will take it from there.
So we implement the logic in our webservice operation:

    /**
     * This is the last solution displayed that is the most simple
     */
    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        Object mailBody = context.createProducerTemplate().sendBody("direct:start", parameters);
        System.out.println("Body:" + mailBody);

        // return an OK reply
        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

Notice that we get the producer template using the createProducerTemplate method on the CamelContext. Then we send the input parameters to the direct:start endpoint and it will route it to the velocity endpoint that will generate the mail body. Since we use direct as the consumer endpoint (=from) and its a synchronous exchange we will get the response back from the route. And the response is of course the output from the velocity endpoint.

About creating ProducerTemplate

In the example above we create a new ProducerTemplate when the reportIncident method is invoked. However in reality you should only create the template once and re-use it. See this FAQ entry.

We have now completed this part of the picture:

Unit testing

Now is the time we would like to unit test what we got now. So we call for camel and its great test kit. For this to work we need to add it to the pom.xml

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-core</artifactId>
            <version>1.4.0</version>
            <scope>test</scope>
            <type>test-jar</type>
        </dependency>

After adding it to the pom.xml you should refresh your Java Editor so it pickups the new jar. Then we are ready to create out unit test class.
We create this unit test skeleton, where we extend this class ContextTestSupport

package org.apache.camel.example.reportincident;

import org.apache.camel.ContextTestSupport;
import org.apache.camel.builder.RouteBuilder;

/**
 * Unit test of our routes
 */
public class ReportIncidentRoutesTest extends ContextTestSupport {

}

ContextTestSupport is a supporting unit test class for much easier unit testing with Apache Camel. The class is extending JUnit TestCase itself so you get all its glory. What we need to do now is to somehow tell this unit test class that it should use our route builder as this is the one we gonna test. So we do this by implementing the createRouteBuilder method.

    @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new ReportIncidentRoutes();
    }

That is easy just return an instance of our route builder and this unit test will use our routes.

It is quite common in Camel itself to unit test using routes defined as an anonymous inner class, such as illustrated below:

    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {
            public void configure() throws Exception {
                // TODO: Add your routes here, such as:
                from("jms:queue:inbox").to("file://target/out");
            }
        };
    }

The same technique is of course also possible for end-users of Camel to create parts of your routes and test them separately in many test classes.
However in this tutorial we test the real route that is to be used for production, so we just return an instance of the real one.

We then code our unit test method that sends a message to the route and assert that its transformed to the mail body using the Velocity template.

    public void testTransformMailBody() throws Exception {
        // create a dummy input with some input data
        InputReportIncident parameters = createInput();

        // send the message (using the sendBody method that takes a parameters as the input body)
        // to "direct:start" that kick-starts the route
        // the response is returned as the out object, and its also the body of the response
        Object out = context.createProducerTemplate().sendBody("direct:start", parameters);

        // convert the response to a string using camel converters. However we could also have casted it to
        // a string directly but using the type converters ensure that Camel can convert it if it wasn't a string
        // in the first place. The type converters in Camel is really powerful and you will later learn to
        // appreciate them and wonder why its not build in Java out-of-the-box
        String body = context.getTypeConverter().convertTo(String.class, out);

        // do some simple assertions of the mail body
        assertTrue(body.startsWith("Incident 123 has been reported on the 2008-07-16 by Claus Ibsen."));
    }

    /**
     * Creates a dummy request to be used for input
     */
    protected InputReportIncident createInput() {
        InputReportIncident input = new InputReportIncident();
        input.setIncidentId("123");
        input.setIncidentDate("2008-07-16");
        input.setGivenName("Claus");
        input.setFamilyName("Ibsen");
        input.setSummary("bla bla");
        input.setDetails("more bla bla");
        input.setEmail("davsclaus@apache.org");
        input.setPhone("+45 2962 7576");
        return input;
    }

Adding the File Backup

The next piece of puzzle that is missing is to store the mail body as a backup file. So we turn back to our route and the EIP patterns. We use the Pipes and Filters pattern here to chain the routing as:

    public void configure() throws Exception {
        from("direct:start")
            .to("velocity:MailBody.vm")
            // using pipes-and-filters we send the output from the previous to the next
            .to("file://target/subfolder");
     }

Notice that we just add a 2nd .to on the newline. Camel will default use the Pipes and Filters pattern here when there are multi endpoints chained liked this. We could have used the pipeline verb to let out stand out that its the Pipes and Filters pattern such as:

        from("direct:start")
            // using pipes-and-filters we send the output from the previous to the next
            .pipeline("velocity:MailBody.vm", "file://target/subfolder");

But most people are using the multi .to style instead.

We re-run out unit test and verifies that it still passes:

Running org.apache.camel.example.reportincident.ReportIncidentRoutesTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.157 sec

But hey we have added the file producer endpoint and thus a file should also be created as the backup file. If we look in the target/subfolder we can see that something happened.
On my humble laptop it created this folder: target\subfolder\ID-claus-acer. So the file producer create a sub folder named ID-claus-acer what is this? Well Camel auto generates an unique filename based on the unique message id if not given instructions to use a fixed filename. In fact it creates another sub folder and name the file as: target\subfolder\ID-claus-acer\3750-1219148558921\1-0 where 1-0 is the file with the mail body. What we want is to use our own filename instead of this auto generated filename. This is archived by adding a header to the message with the filename to use. So we need to add this to our route and compute the filename based on the message content.

Setting the filename

For starters we show the simple solution and build from there. We start by setting a constant filename, just to verify that we are on the right path, to instruct the file producer what filename to use. The file producer uses a special header FileComponent.HEADER_FILE_NAME to set the filename.

What we do is to send the header when we "kick-start" the routing as the header will be propagated from the direct queue to the file producer. What we need to do is to use the ProducerTemplate.sendBodyAndHeader method that takes both a body and a header. So we change out webservice code to include the filename also:

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        // create the producer template to use for sending messages
        ProducerTemplate producer = context.createProducerTemplate();
        // send the body and the filename defined with the special header key 
        Object mailBody = producer.sendBodyAndHeader("direct:start", parameters, FileComponent.HEADER_FILE_NAME, "incident.txt");
        System.out.println("Body:" + mailBody);

        // return an OK reply
        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

However we could also have used the route builder itself to configure the constant filename as shown below:

    public void configure() throws Exception {
        from("direct:start")
            .to("velocity:MailBody.vm")
            // set the filename to a constant before the file producer receives the message
            .setHeader(FileComponent.HEADER_FILE_NAME, constant("incident.txt"))
            .to("file://target/subfolder");
     }

But Camel can be smarter and we want to dynamic set the filename based on some of the input parameters, how can we do this?
Well the obvious solution is to compute and set the filename from the webservice implementation, but then the webservice implementation has such logic and we want this decoupled, so we could create our own POJO bean that has a method to compute the filename. We could then instruct the routing to invoke this method to get the computed filename. This is a string feature in Camel, its Bean binding. So lets show how this can be done:

Using Bean Language to compute the filename

First we create our plain java class that computes the filename, and it has 100% no dependencies to Camel what so ever.

/**
 * Plain java class to be used for filename generation based on the reported incident
 */
public class FilenameGenerator {

    public String generateFilename(InputReportIncident input) {
        // compute the filename
        return "incident-" + input.getIncidentId() + ".txt";
    }

}

The class is very simple and we could easily create unit tests for it to verify that it works as expected. So what we want now is to let Camel invoke this class and its generateFilename with the input parameters and use the output as the filename. Pheeeww is this really possible out-of-the-box in Camel? Yes it is. So lets get on with the show. We have the code that computes the filename, we just need to call it from our route using the Bean Language:

    public void configure() throws Exception {
        from("direct:start")
            // set the filename using the bean language and call the FilenameGenerator class.
            // the 2nd null parameter is optional methodname, to be used to avoid ambiguity.
            // if not provided Camel will try to figure out the best method to invoke, as we
            // only have one method this is very simple
            .setHeader(FileComponent.HEADER_FILE_NAME, BeanLanguage.bean(FilenameGenerator.class, null))
            .to("velocity:MailBody.vm")
            .to("file://target/subfolder");
    }

Notice that we use the bean language where we supply the class with our bean to invoke. Camel will instantiate an instance of the class and invoke the suited method. For completeness and ease of code readability we add the method name as the 2nd parameter

            .setHeader(FileComponent.HEADER_FILE_NAME, BeanLanguage.bean(FilenameGenerator.class, "generateFilename"))

Then other developers can understand what the parameter is, instead of null.

Now we have a nice solution, but as a sidetrack I want to demonstrate the Camel has other languages out-of-the-box, and that scripting language is a first class citizen in Camel where it etc. can be used in content based routing. However we want it to be used for the filename generation.

Using a script language to set the filename

We could do as in the previous parts where we send the computed filename as a message header when we "kick-start" the route. But we want to learn new stuff so we look for a different solution using some of Camels many Languages. As OGNL is a favorite language of mine (used by WebWork) so we pick this baby for a Camel ride. For starters we must add it to our pom.xml:

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-ognl</artifactId>
            <version>${camel-version}</version>
        </dependency>

And remember to refresh your editor so you got the new .jars.
We want to construct the filename based on this syntax: mail-incident-#ID#.txt where #ID# is the incident id from the input parameters. As OGNL is a language that can invoke methods on bean we can invoke the getIncidentId() on the message body and then concat it with the fixed pre and postfix strings.

In OGNL glory this is done as:

"'mail-incident-' + request.body.incidentId + '.txt'"

where request.body.incidentId computes to:

  • request is the IN message. See the OGNL for other predefined objects available
  • body is the body of the in message
  • incidentId will invoke the getIncidentId() method on the body.
    The rest is just more or less regular plain code where we can concat strings.

Now we got the expression to dynamic compute the filename on the fly we need to set it on our route so we turn back to our route, where we can add the OGNL expression:

    public void configure() throws Exception {
        from("direct:start")
            // we need to set the filename and uses OGNL for this
            .setHeader(FileComponent.HEADER_FILE_NAME, OgnlExpression.ognl("'mail-incident-' + request.body.incidentId + '.txt'"))
            // using pipes-and-filters we send the output from the previous to the next
            .pipeline("velocity:MailBody.vm", "file://target/subfolder");
    }

And since we are on Java 1.5 we can use the static import of ognl so we have:

import static org.apache.camel.language.ognl.OgnlExpression.ognl;
...
    .setHeader(FileComponent.HEADER_FILE_NAME, ognl("'mail-incident-' + request.body.incidentId + '.txt'"))

Notice the import static also applies for all the other languages, such as the Bean Language we used previously.

Whatever worked for you we have now implemented the backup of the data files:

Sending the email

What we need to do before the solution is completed is to actually send the email with the mail body we generated and stored as a file. In the previous part we did this with a File consumer, that we manually added to the CamelContext. We can do this quite easily with the routing.

import org.apache.camel.builder.RouteBuilder;

public class ReportIncidentRoutes extends RouteBuilder {

    public void configure() throws Exception {
        // first part from the webservice -> file backup
        from("direct:start")
            .setHeader(FileComponent.HEADER_FILE_NAME, bean(FilenameGenerator.class, "generateFilename"))
            .to("velocity:MailBody.vm")
            .to("file://target/subfolder");

        // second part from the file backup -> send email
        from("file://target/subfolder")
            // set the subject of the email
            .setHeader("subject", constant("new incident reported"))
            // send the email
            .to("smtp://someone@localhost?password=secret&to=incident@mycompany.com");
    }

}

The last 3 lines of code does all this. It adds a file consumer from("file://target/subfolder"), sets the mail subject, and finally send it as an email.

The DSL is really powerful where you can express your routing integration logic.
So we completed the last piece in the picture puzzle with just 3 lines of code.

We have now completed the integration:

Conclusion

We have just briefly touched the routing in Camel and shown how to implement them using the fluent builder syntax in Java. There is much more to the routing in Camel than shown here, but we are learning step by step. We continue in part 5. See you there.

Links

Better JMS Transport for CXF Webservice using Apache Camel

Configuring JMS in Apache CXF before Version 2.1.3 is possible but not really easy or nice. This article shows how to use Apache Camel to provide a better JMS Transport for CXF.

Update: Since CXF 2.1.3 there is a new way of configuring JMS (Using the JMSConfigFeature). It makes JMS config for CXF as easy as with Camel. Using Camel for JMS is still a good idea if you want to use the rich feature of Camel for routing and other Integration Scenarios that CXF does not support.

You can find the original announcement for this Tutorial and some additional info on Christian Schneider´s Blog

So how to connect Apache Camel and CXF

The best way to connect Camel and CXF is using the Camel transport for CXF. This is a camel module that registers with cxf as a new transport. It is quite easy to configure.

<bean class="org.apache.camel.component.cxf.transport.CamelTransportFactory">
  <property name="bus" ref="cxf" />
  <property name="camelContext" ref="camelContext" />
  <property name="transportIds">
    <list>
      <value>http://cxf.apache.org/transports/camel</value>
    </list>
  </property>
</bean>

This bean registers with CXF and provides a new transport prefix camel:// that can be used in CXF address configurations. The bean references a bean cxf which will be already present in your config. The other refrenceis a camel context. We will later define this bean to provide the routing config.

How is JMS configured in Camel

In camel you need two things to configure JMS. A ConnectionFactory and a JMSComponent. As ConnectionFactory you can simply set up the normal Factory your JMS provider offers or bind a JNDI ConnectionFactory. In this example we use the ConnectionFactory provided by ActiveMQ.

<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
  <property name="brokerURL" value="tcp://localhost:61616" />
</bean>

Then we set up the JMSComponent. It offers a new transport prefix to camel that we simply call jms. If we need several JMSComponents we can differentiate them by their name.

<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
  <property name="connectionFactory" ref="jmsConnectionFactory" />
  <property name="useMessageIDAsCorrelationID" value="true" />
</bean>

You can find more details about the JMSComponent at the Camel Wiki. For example you find the complete configuration options and a JNDI sample there.

Setting up the CXF client

We will configure a simple CXF webservice client. It will use stub code generated from a wsdl. The webservice client will be configured to use JMS directly. You can also use a direct: Endpoint and do the routing to JMS in the Camel Context.

<client id="CustomerService" xmlns="http://cxf.apache.org/jaxws" xmlns:customer="http://customerservice.example.com/"
  serviceName="customer:CustomerServiceService"
  endpointName="customer:CustomerServiceEndpoint"
  address="camel:jms:queue:CustomerService"
  serviceClass="com.example.customerservice.CustomerService">
</client>

We explicitly configure serviceName and endpointName so they are not read from the wsdl. The names we use are arbitrary and have no further function but we set them to look nice. The serviceclass points to the service interface that was generated from the wsdl. Now the important thing is address. Here we tell cxf to use the camel transport, use the JmsComponent who registered the prefix "jms" and use the queue "CustomerService".

Setting up the CamelContext

As we do not need additional routing an empty CamelContext bean will suffice.

<camelContext id="camelContext" xmlns="http://activemq.apache.org/camel/schema/spring">
</camelContext>

Running the Example

  • Follow the readme.txt

Conclusion

As you have seen in this example you can use Camel to connect services to JMS easily while being able to also use the rich integration features of Apache Camel.

Tutorial using Axis 1.4 with Apache Camel

Removed from distribution

This example has been removed from Camel 2.9 onwards. Apache Axis 1.4 is a very old and unsupported framework. We encourage users to use CXF instead of Axis.

Prerequisites

This tutorial uses Maven 2 to setup the Camel project and for dependencies for artifacts.

Distribution

This sample is distributed with the Camel 1.5 distribution as examples/camel-example-axis.

Introduction

Apache Axis is/was widely used as a webservice framework. So in line with some of the other tutorials to demonstrate how Camel is not an invasive framework but is flexible and integrates well with existing solution.

We have an existing solution that exposes a webservice using Axis 1.4 deployed as web applications. This is a common solution. We use contract first so we have Axis generated source code from an existing wsdl file. Then we show how we introduce Spring and Camel to integrate with Axis.

This tutorial uses the following frameworks:

  • Maven 2.0.9
  • Apache Camel 1.5.0
  • Apache Axis 1.4
  • Spring 2.5.5

Setting up the project to run Axis

This first part is about getting the project up to speed with Axis. We are not touching Camel or Spring at this time.

Maven 2

Axis dependencies is available for maven 2 so we configure our pom.xml as:

        <dependency>
            <groupId>org.apache.axis</groupId>
            <artifactId>axis</artifactId>
            <version>1.4</version>
        </dependency>

        <dependency>
            <groupId>org.apache.axis</groupId>
            <artifactId>axis-jaxrpc</artifactId>
            <version>1.4</version>
        </dependency>

        <dependency>
            <groupId>org.apache.axis</groupId>
            <artifactId>axis-saaj</artifactId>
            <version>1.4</version>
        </dependency>

	<dependency>
	    <groupId>axis</groupId>
	    <artifactId>axis-wsdl4j</artifactId>
	    <version>1.5.1</version>
	</dependency>

	<dependency>
	    <groupId>commons-discovery</groupId>
	    <artifactId>commons-discovery</artifactId>
	    <version>0.4</version>
	</dependency> 

        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.14</version>
        </dependency>

Then we need to configure maven to use Java 1.5 and the Axis maven plugin that generates the source code based on the wsdl file:

<!-- to compile with 1.5 -->
	<plugin>
		<groupId>org.apache.maven.plugins</groupId>
		<artifactId>maven-compiler-plugin</artifactId>
		<configuration>
			<source>1.5</source>
			<target>1.5</target>
		</configuration>
	</plugin>

            <plugin>
               <groupId>org.codehaus.mojo</groupId>
               <artifactId>axistools-maven-plugin</artifactId>
               <configuration>
	          <sourceDirectory>src/main/resources/</sourceDirectory>
                  <packageSpace>com.mycompany.myschema</packageSpace>
                  <testCases>false</testCases>
                  <serverSide>true</serverSide>
                  <subPackageByFileName>false</subPackageByFileName>
               </configuration>
               <executions>
                 <execution>
                   <goals>
                     <goal>wsdl2java</goal>
                   </goals>
                 </execution>
               </executions>
            </plugin>

wsdl

We use the same .wsdl file as the Tutorial-Example-ReportIncident and copy it to src/main/webapp/WEB-INF/wsdl

<?xml version="1.0" encoding="ISO-8859-1"?>
<wsdl:definitions xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
	xmlns:tns="http://reportincident.example.camel.apache.org"
	xmlns:xs="http://www.w3.org/2001/XMLSchema"
	xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
	xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
	targetNamespace="http://reportincident.example.camel.apache.org">

	<!-- Type definitions for input- and output parameters for webservice -->
	<wsdl:types>
	<xs:schema targetNamespace="http://reportincident.example.camel.apache.org">
			<xs:element name="inputReportIncident">
				<xs:complexType>
					<xs:sequence>
						<xs:element type="xs:string"  name="incidentId"/>
						<xs:element type="xs:string"  name="incidentDate"/>
						<xs:element type="xs:string"  name="givenName"/>
						<xs:element type="xs:string"  name="familyName"/>
						<xs:element type="xs:string"  name="summary"/>
						<xs:element type="xs:string"  name="details"/>
						<xs:element type="xs:string"  name="email"/>
						<xs:element type="xs:string"  name="phone"/>
					</xs:sequence>
				</xs:complexType>
			</xs:element>
			<xs:element name="outputReportIncident">
				<xs:complexType>
					<xs:sequence>
						<xs:element type="xs:string" name="code"/>
					</xs:sequence>
				</xs:complexType>
			</xs:element>
		</xs:schema>
	</wsdl:types>

	<!-- Message definitions for input and output -->
	<wsdl:message name="inputReportIncident">
		<wsdl:part name="parameters" element="tns:inputReportIncident"/>
	</wsdl:message>
	<wsdl:message name="outputReportIncident">
		<wsdl:part name="parameters" element="tns:outputReportIncident"/>
	</wsdl:message>

	<!-- Port (interface) definitions -->
	<wsdl:portType name="ReportIncidentEndpoint">
		<wsdl:operation name="ReportIncident">
			<wsdl:input message="tns:inputReportIncident"/>
			<wsdl:output message="tns:outputReportIncident"/>
		</wsdl:operation>
	</wsdl:portType>

	<!-- Port bindings to transports and encoding - HTTP, document literal encoding is used -->
	<wsdl:binding name="ReportIncidentBinding" type="tns:ReportIncidentEndpoint">
		<soap:binding transport="http://schemas.xmlsoap.org/soap/http"/>
		<wsdl:operation name="ReportIncident">
			<soap:operation
				soapAction="http://reportincident.example.camel.apache.org/ReportIncident"
				style="document"/>
			<wsdl:input>
				<soap:body parts="parameters" use="literal"/>
			</wsdl:input>
			<wsdl:output>
				<soap:body parts="parameters" use="literal"/>
			</wsdl:output>
		</wsdl:operation>
	</wsdl:binding>

	<!-- Service definition -->
	<wsdl:service name="ReportIncidentService">
		<wsdl:port name="ReportIncidentPort" binding="tns:ReportIncidentBinding">
			<soap:address location="http://reportincident.example.camel.apache.org"/>
		</wsdl:port>
	</wsdl:service>

</wsdl:definitions>

Configuring Axis

Okay we are now setup for the contract first development and can generate the source file. For now we are still only using standard Axis and not Spring nor Camel. We still need to setup Axis as a web application so we configure the web.xml in src/main/webapp/WEB-INF/web.xml as:

    <servlet>
        <servlet-name>axis</servlet-name>
        <servlet-class>org.apache.axis.transport.http.AxisServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>axis</servlet-name>
        <url-pattern>/services/*</url-pattern>
    </servlet-mapping>

The web.xml just registers Axis servlet that is handling the incoming web requests to its servlet mapping. We still need to configure Axis itself and this is done using its special configuration file server-config.wsdd. We nearly get this file for free if we let Axis generate the source code so we run the maven goal:

mvn axistools:wsdl2java

The tool will generate the source code based on the wsdl and save the files to the following folder:

.\target\generated-sources\axistools\wsdl2java\org\apache\camel\example\reportincident
deploy.wsdd
InputReportIncident.java
OutputReportIncident.java
ReportIncidentBindingImpl.java
ReportIncidentBindingStub.java
ReportIncidentService_PortType.java
ReportIncidentService_Service.java
ReportIncidentService_ServiceLocator.java
undeploy.wsdd

This is standard Axis and so far no Camel or Spring has been touched. To implement our webservice we will add our code, so we create a new class AxisReportIncidentService that implements the port type interface where we can implement our code logic what happens when the webservice is invoked.

package org.apache.camel.example.axis;

import org.apache.camel.example.reportincident.InputReportIncident;
import org.apache.camel.example.reportincident.OutputReportIncident;
import org.apache.camel.example.reportincident.ReportIncidentService_PortType;

import java.rmi.RemoteException;

/**
 * Axis webservice
 */
public class AxisReportIncidentService implements ReportIncidentService_PortType {

    public OutputReportIncident reportIncident(InputReportIncident parameters) throws RemoteException {
        System.out.println("Hello AxisReportIncidentService is called from " + parameters.getGivenName());

        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

}

Now we need to configure Axis itself and this is done using its server-config.wsdd file. We nearly get this for for free from the auto generated code, we copy the stuff from deploy.wsdd and made a few modifications:

<?xml version="1.0" encoding="UTF-8"?>
<deployment xmlns="http://xml.apache.org/axis/wsdd/" xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
  <!-- global configuration -->
	<globalConfiguration>
		<parameter name="sendXsiTypes" value="true"/>
		<parameter name="sendMultiRefs" value="true"/>
		<parameter name="sendXMLDeclaration" value="true"/>
		<parameter name="axis.sendMinimizedElements" value="true"/>
	</globalConfiguration>
	<handler name="URLMapper" type="java:org.apache.axis.handlers.http.URLMapper"/>

  <!-- this service is from deploy.wsdd -->
  <service name="ReportIncidentPort" provider="java:RPC" style="document" use="literal">
      <parameter name="wsdlTargetNamespace" value="http://reportincident.example.camel.apache.org"/>
      <parameter name="wsdlServiceElement" value="ReportIncidentService"/>
      <parameter name="schemaUnqualified" value="http://reportincident.example.camel.apache.org"/>
      <parameter name="wsdlServicePort" value="ReportIncidentPort"/>
      <parameter name="className" value="org.apache.camel.example.reportincident.ReportIncidentBindingImpl"/>
      <parameter name="wsdlPortType" value="ReportIncidentService"/>
      <parameter name="typeMappingVersion" value="1.2"/>
      <operation name="reportIncident" qname="ReportIncident" returnQName="retNS:outputReportIncident" xmlns:retNS="http://reportincident.example.camel.apache.org"
                 returnType="rtns:>outputReportIncident" xmlns:rtns="http://reportincident.example.camel.apache.org"
                 soapAction="http://reportincident.example.camel.apache.org/ReportIncident" >
        <parameter qname="pns:inputReportIncident" xmlns:pns="http://reportincident.example.camel.apache.org"
                 type="tns:>inputReportIncident" xmlns:tns="http://reportincident.example.camel.apache.org"/>
      </operation>
      <parameter name="allowedMethods" value="reportIncident"/>

      <typeMapping
        xmlns:ns="http://reportincident.example.camel.apache.org"
        qname="ns:>outputReportIncident"
        type="java:org.apache.camel.example.reportincident.OutputReportIncident"
        serializer="org.apache.axis.encoding.ser.BeanSerializerFactory"
        deserializer="org.apache.axis.encoding.ser.BeanDeserializerFactory"
        encodingStyle=""
      />
      <typeMapping
        xmlns:ns="http://reportincident.example.camel.apache.org"
        qname="ns:>inputReportIncident"
        type="java:org.apache.camel.example.reportincident.InputReportIncident"
        serializer="org.apache.axis.encoding.ser.BeanSerializerFactory"
        deserializer="org.apache.axis.encoding.ser.BeanDeserializerFactory"
        encodingStyle=""
      />
  </service>

  <!-- part of Axis configuration -->
	<transport name="http">
		<requestFlow>
			<handler type="URLMapper"/>
			<handler type="java:org.apache.axis.handlers.http.HTTPAuthHandler"/>
		</requestFlow>
	</transport>
</deployment>

The globalConfiguration and transport is not in the deploy.wsdd file so you gotta write that yourself. The service is a 100% copy from deploy.wsdd. Axis has more configuration to it than shown here, but then you should check the Axis documentation.

What we need to do now is important, as we need to modify the above configuration to use our webservice class than the default one, so we change the classname parameter to our class AxisReportIncidentService:

<parameter name="className" value="org.apache.camel.example.axis.AxisReportIncidentService"/>

Running the Example

Now we are ready to run our example for the first time, so we use Jetty as the quick web container using its maven command:

mvn jetty:run

Then we can hit the web browser and enter this URL: http://localhost:8080/camel-example-axis/services and you should see the famous Axis start page with the text And now... Some Services.

Clicking on the .wsdl link shows the wsdl file, but what. It's an auto generated one and not our original .wsdl file. So we need to fix this ASAP and this is done by configuring Axis in the server-config.wsdd file:

  <service name="ReportIncidentPort" provider="java:RPC" style="document" use="literal">
    <wsdlFile>/WEB-INF/wsdl/report_incident.wsdl</wsdlFile>
    ...

We do this by adding the wsdlFile tag in the service element where we can point to the real .wsdl file.

Integrating Spring

First we need to add its dependencies to the pom.xml.

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>2.5.5</version>
        </dependency>

Spring is integrated just as it would like to, we add its listener to the web.xml and a context parameter to be able to configure precisely what spring xml files to use:

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>
            classpath:axis-example-context.xml
        </param-value>
    </context-param>

    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

Next is to add a plain spring XML file named axis-example-context.xml in the src/main/resources folder.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
         http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

</beans>

The spring XML file is currently empty. We hit jetty again with mvn jetty:run just to make sure Spring was setup correctly.

Using Spring

We would like to be able to get hold of the Spring ApplicationContext from our webservice so we can get access to the glory spring, but how do we do this? And our webservice class AxisReportIncidentService is created and managed by Axis we want to let Spring do this. So we have two problems.

We solve these problems by creating a delegate class that Axis creates, and this delegate class gets hold on Spring and then gets our real webservice as a spring bean and invoke the service.

First we create a new class that is 100% independent from Axis and just a plain POJO. This is our real service.

package org.apache.camel.example.axis;

import org.apache.camel.example.reportincident.InputReportIncident;
import org.apache.camel.example.reportincident.OutputReportIncident;

/**
 * Our real service that is not tied to Axis
 */
public class ReportIncidentService  {

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        System.out.println("Hello ReportIncidentService is called from " + parameters.getGivenName());

        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

}

So now we need to get from AxisReportIncidentService to this one ReportIncidentService using Spring. Well first of all we add our real service to spring XML configuration file so Spring can handle its lifecycle:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
         http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
    
    <bean id="incidentservice" class="org.apache.camel.example.axis.ReportIncidentService"/>

</beans>

And then we need to modify AxisReportIncidentService to use Spring to lookup the spring bean id="incidentservice" and delegate the call. We do this by extending the spring class org.springframework.remoting.jaxrpc.ServletEndpointSupport so the refactored code is:

package org.apache.camel.example.axis;

import org.apache.camel.example.reportincident.InputReportIncident;
import org.apache.camel.example.reportincident.OutputReportIncident;
import org.apache.camel.example.reportincident.ReportIncidentService_PortType;
import org.springframework.remoting.jaxrpc.ServletEndpointSupport;

import java.rmi.RemoteException;

/**
 * Axis webservice
 */
public class AxisReportIncidentService extends ServletEndpointSupport implements ReportIncidentService_PortType {

    public OutputReportIncident reportIncident(InputReportIncident parameters) throws RemoteException {
        // get hold of the spring bean from the application context
        ReportIncidentService service = (ReportIncidentService) getApplicationContext().getBean("incidentservice");

        // delegate to the real service
        return service.reportIncident(parameters);
    }

}

To see if everything is okay we run mvn jetty:run.

In the code above we get hold of our service at each request by looking up in the application context. However Spring also supports an init method where we can do this once. So we change the code to:

public class AxisReportIncidentService extends ServletEndpointSupport implements ReportIncidentService_PortType {

    private ReportIncidentService service;

    @Override
    protected void onInit() throws ServiceException {
        // get hold of the spring bean from the application context
        service = (ReportIncidentService) getApplicationContext().getBean("incidentservice");
    }

    public OutputReportIncident reportIncident(InputReportIncident parameters) throws RemoteException {
        // delegate to the real service
        return service.reportIncident(parameters);
    }

}

So now we have integrated Axis with Spring and we are ready for Camel.

Integrating Camel

Again the first step is to add the dependencies to the maven pom.xml file:

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-core</artifactId>
            <version>1.5.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-spring</artifactId>
            <version>1.5.0</version>
        </dependency>

Now that we have integrated with Spring then we easily integrate with Camel as Camel works well with Spring.

Camel does not require Spring

Camel does not require Spring, we could easily have used Camel without Spring, but most users prefer to use Spring also.

We choose to integrate Camel in the Spring XML file so we add the camel namespace and the schema location:

xmlns:camel="http://activemq.apache.org/camel/schema/spring"
http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd"

CamelContext

CamelContext is the heart of Camel its where all the routes, endpoints, components, etc. is registered. So we setup a CamelContext and the spring XML files looks like:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:camel="http://activemq.apache.org/camel/schema/spring"
       xsi:schemaLocation="
         http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
         http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">

    <bean id="incidentservice" class="org.apache.camel.example.axis.ReportIncidentService"/>

    <camel:camelContext id="camel">
        <!-- TODO: Here we can add Camel stuff -->
    </camel:camelContext>

</beans>

Store a file backup

We want to store the web service request as a file before we return a response. To do this we want to send the file content as a message to an endpoint that produces the file. So we need to do two steps:

  • configure the file backup endpoint
  • send the message to the endpoint

The endpoint is configured in spring XML so we just add it as:

    <camel:camelContext id="camelContext">
        <!-- endpoint named backup that is configued as a file component -->
        <camel:endpoint id="backup" uri="file://target?append=false"/>
    </camel:camelContext>

In the CamelContext we have defined our endpoint with the id backup and configured it use the URL notation that we know from the internet. Its a file scheme that accepts a context and some options. The contest is target and its the folder to store the file. The option is just as the internet with ? and & for subsequent options. We configure it to not append, meaning than any existing file will be overwritten. See the File component for options and how to use the camel file endpoint.

Next up is to be able to send a message to this endpoint. The easiest way is to use a ProducerTemplate. A ProducerTemplate is inspired by Spring template pattern with for instance JmsTemplate or JdbcTemplate in mind. The template that all the grunt work and exposes a simple interface to the end-user where he/she can set the payload to send. Then the template will do proper resource handling and all related issues in that regard. But how do we get hold of such a template? Well the CamelContext is able to provide one. This is done by configuring the template on the camel context in the spring XML as:

    <camel:camelContext id="camelContext">
        <!-- producer template exposed with this id -->
        <camel:template id="camelTemplate"/>

        <!-- endpoint named backup that is configued as a file component -->
        <camel:endpoint id="backup" uri="file://target?append=false"/>
    </camel:camelContext>

Then we can expose a ProducerTemplate property on our service with a setter in the Java code as:

public class ReportIncidentService {

    private ProducerTemplate template;

    public void setTemplate(ProducerTemplate template) {
        this.template = template;
    }

And then let Spring handle the dependency inject as below:

    <bean id="incidentservice" class="org.apache.camel.example.axis.ReportIncidentService">
        <!-- set the producer template to use from the camel context below -->
        <property name="template" ref="camelTemplate"/>
    </bean>

Now we are ready to use the producer template in our service to send the payload to the endpoint. The template has many sendXXX methods for this purpose. But before we send the payload to the file endpoint we must also specify what filename to store the file as. This is done by sending meta data with the payload. In Camel metadata is sent as headers. Headers is just a plain Map<String, Object>. So if we needed to send several metadata then we could construct an ordinary HashMap and put the values in there. But as we just need to send one header with the filename Camel has a convenient send method sendBodyAndHeader so we choose this one.

    public OutputReportIncident reportIncident(InputReportIncident parameters) {
        System.out.println("Hello ReportIncidentService is called from " + parameters.getGivenName());

        String data = parameters.getDetails();

        // store the data as a file
        String filename = parameters.getIncidentId() + ".txt";
        // send the data to the endpoint and the header contains what filename it should be stored as
        template.sendBodyAndHeader("backup", data, "org.apache.camel.file.name", filename);

        OutputReportIncident out = new OutputReportIncident();
        out.setCode("OK");
        return out;
    }

The template in the code above uses 4 parameters:

  • the endpoint name, in this case the id referring to the endpoint defined in Spring XML in the camelContext element.
  • the payload, can be any kind of object
  • the key for the header, in this case a Camel keyword to set the filename
  • and the value for the header

Running the example

We start our integration with maven using mvn jetty:run. Then we open a browser and hit http://localhost:8080. Jetty is so smart that it display a frontpage with links to the deployed application so just hit the link and you get our application. Now we hit append /services to the URL to access the Axis frontpage. The URL should be http://localhost:8080/camel-example-axis/services.

You can then test it using a web service test tools such as SoapUI.
Hitting the service will output to the console

2008-09-06 15:01:41.718::INFO:  Started SelectChannelConnector @ 0.0.0.0:8080
[INFO] Started Jetty Server
Hello ReportIncidentService is called from Ibsen

And there should be a file in the target subfolder.

dir target /b
123.txt

Unit Testing

We would like to be able to unit test our ReportIncidentService class. So we add junit to the maven dependency:

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.2</version>
            <scope>test</scope>
        </dependency>

And then we create a plain junit testcase for our service class.

package org.apache.camel.example.axis;

import junit.framework.TestCase;
import org.apache.camel.example.reportincident.InputReportIncident;
import org.apache.camel.example.reportincident.OutputReportIncident;

/**
 * Unit test of service
 */
public class ReportIncidentServiceTest extends TestCase {

    public void testIncident() {
        ReportIncidentService service = new ReportIncidentService();

        InputReportIncident input = createDummyIncident();
        OutputReportIncident output = service.reportIncident(input);
        assertEquals("OK", output.getCode());
    }

   protected InputReportIncident createDummyIncident() {
        InputReportIncident input = new InputReportIncident();
        input.setEmail("davsclaus@apache.org");
        input.setIncidentId("12345678");
        input.setIncidentDate("2008-07-13");
        input.setPhone("+45 2962 7576");
        input.setSummary("Failed operation");
        input.setDetails("The wrong foot was operated.");
        input.setFamilyName("Ibsen");
        input.setGivenName("Claus");
        return input;
    }

}

Then we can run the test with maven using: mvn test. But we will get a failure:

Running org.apache.camel.example.axis.ReportIncidentServiceTest
Hello ReportIncidentService is called from Claus
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.235 sec <<< FAILURE!

Results :

Tests in error:
  testIncident(org.apache.camel.example.axis.ReportIncidentServiceTest)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

What is the problem? Well our service uses a CamelProducer (the template) to send a message to the file endpoint so the message will be stored in a file. What we need is to get hold of such a producer and inject it on our service, by calling the setter.

Since Camel is very light weight and embedable we are able to create a CamelContext and add the endpoint in our unit test code directly. We do this to show how this is possible:

    private CamelContext context;

    @Override
    protected void setUp() throws Exception {
        super.setUp();
        // CamelContext is just created like this
        context = new DefaultCamelContext();

        // then we can create our endpoint and set the options
        FileEndpoint endpoint = new FileEndpoint();
        // the endpoint must have the camel context set also
        endpoint.setCamelContext(context);
        // our output folder
        endpoint.setFile(new File("target"));
        // and the option not to append
        endpoint.setAppend(false);

        // then we add the endpoint just in java code just as the spring XML, we register it with the "backup" id.
        context.addSingletonEndpoint("backup", endpoint);

        // finally we need to start the context so Camel is ready to rock
        context.start();
    }

    @Override
    protected void tearDown() throws Exception {
        super.tearDown();
        // and we are nice boys so we stop it to allow resources to clean up
        context.stop();
    }

So now we are ready to set the ProducerTemplate on our service, and we get a hold of that baby from the CamelContext as:

    public void testIncident() {
        ReportIncidentService service = new ReportIncidentService();

        // get a producer template from the camel context
        ProducerTemplate template = context.createProducerTemplate();
        // inject it on our service using the setter
        service.setTemplate(template);

        InputReportIncident input = createDummyIncident();
        OutputReportIncident output = service.reportIncident(input);
        assertEquals("OK", output.getCode());
    }

And this time when we run the unit test its a success:

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

We would like to test that the file exists so we add these two lines to our test method:

        // should generate a file also
        File file = new File("target/" + input.getIncidentId() + ".txt");
        assertTrue("File should exists", file.exists());

Smarter Unit Testing with Spring

The unit test above requires us to assemble the Camel pieces manually in java code. What if we would like our unit test to use our spring configuration file axis-example-context.xml where we already have setup the endpoint. And of course we would like to test using this configuration file as this is the real file we will use. Well hey presto the xml file is a spring ApplicationContext file and spring is able to load it, so we go the spring path for unit testing. First we add the spring-test jar to our maven dependency:

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <scope>test</scope>
        </dependency>

And then we refactor our unit test to be a standard spring unit class. What we need to do is to extend AbstractJUnit38SpringContextTests instead of TestCase in our unit test. Since Spring 2.5 embraces annotations we will use one as well to instruct what our xml configuration file is located:

@ContextConfiguration(locations = "classpath:axis-example-context.xml")
public class ReportIncidentServiceTest extends AbstractJUnit38SpringContextTests {

What we must remember to add is the classpath: prefix as our xml file is located in src/main/resources. If we omit the prefix then Spring will by default try to locate the xml file in the current package and that is org.apache.camel.example.axis. If the xml file is located outside the classpath you can use file: prefix instead. So with these two modifications we can get rid of all the setup and teardown code we had before and now we will test our real configuration.

The last change is to get hold of the producer template and now we can just refer to the bean id it has in the spring xml file:

        <!-- producer template exposed with this id -->
        <camel:template id="camelTemplate"/>

So we get hold of it by just getting it from the spring ApplicationContext as all spring users is used to do:

        // get a producer template from the the spring context
        ProducerTemplate template = (ProducerTemplate) applicationContext.getBean("camelTemplate");
        // inject it on our service using the setter
        service.setTemplate(template);

Now our unit test is much better, and a real power of Camel is that is fits nicely with Spring and you can use standard Spring'ish unit test to test your Camel applications as well.

Unit Test calling WebService

What if you would like to execute a unit test where you send a webservice request to the AxisReportIncidentService how do we unit test this one? Well first of all the code is merely just a delegate to our real service that we have just tested, but nevertheless its a good question and we would like to know how. Well the answer is that we can exploit that fact that Jetty is also a slim web container that can be embedded anywhere just as Camel can. So we add this to our pom.xml:

        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jetty</artifactId>
            <version>${jetty-version}</version>
            <scope>test</scope>
        </dependency>

Then we can create a new class AxisReportIncidentServiceTest to unit test with Jetty. The code to setup Jetty is shown below with code comments:

public class AxisReportIncidentServiceTest extends TestCase {

    private Server server;

    private void startJetty() throws Exception {
        // create an embedded Jetty server
        server = new Server();

        // add a listener on port 8080 on localhost (127.0.0.1)
        Connector connector = new SelectChannelConnector();
        connector.setPort(8080);
        connector.setHost("127.0.0.1");
        server.addConnector(connector);

        // add our web context path
        WebAppContext wac = new WebAppContext();
        wac.setContextPath("/unittest");
        // set the location of the exploded webapp where WEB-INF is located
        // this is a nice feature of Jetty where we can point to src/main/webapp
        wac.setWar("./src/main/webapp");
        server.setHandler(wac);

        // then start Jetty
        server.setStopAtShutdown(true);
        server.start();
    }

    @Override
    protected void setUp() throws Exception {
        super.setUp();
        startJetty();
    }

    @Override
    protected void tearDown() throws Exception {
        super.tearDown();
        server.stop();
    }

}

Now we just need to send the incident as a webservice request using Axis. So we add the following code:

    public void testReportIncidentWithAxis() throws Exception {
        // the url to the axis webservice exposed by jetty
        URL url = new URL("http://localhost:8080/unittest/services/ReportIncidentPort");

        // Axis stuff to get the port where we can send the webservice request
        ReportIncidentService_ServiceLocator locator = new ReportIncidentService_ServiceLocator();
        ReportIncidentService_PortType port = locator.getReportIncidentPort(url);

        // create input to send
        InputReportIncident input = createDummyIncident();
        // send the webservice and get the response
        OutputReportIncident output = port.reportIncident(input);
        assertEquals("OK", output.getCode());

        // should generate a file also
        File file = new File("target/" + input.getIncidentId() + ".txt");
        assertTrue("File should exists", file.exists());
    }

    protected InputReportIncident createDummyIncident() {
        InputReportIncident input = new InputReportIncident();
        input.setEmail("davsclaus@apache.org");
        input.setIncidentId("12345678");
        input.setIncidentDate("2008-07-13");
        input.setPhone("+45 2962 7576");
        input.setSummary("Failed operation");
        input.setDetails("The wrong foot was operated.");
        input.setFamilyName("Ibsen");
        input.setGivenName("Claus");
        return input;
    }

And now we have an unittest that sends a webservice request using good old Axis.

Annotations

Both Camel and Spring has annotations that can be used to configure and wire trivial settings more elegantly. Camel has the endpoint annotation @EndpointInjected that is just what we need. With this annotation we can inject the endpoint into our service. The annotation takes either a name or uri parameter. The name is the bean id in the Registry. The uri is the URI configuration for the endpoint. Using this you can actually inject an endpoint that you have not defined in the camel context. As we have defined our endpoint with the id backup we use the name parameter.

    @EndpointInject(name = "backup")
    private ProducerTemplate template;

Camel is smart as @EndpointInjected supports different kinds of object types. We like the ProducerTemplate so we just keep it as it is.
Since we use annotations on the field directly we do not need to set the property in the spring xml file so we change our service bean:

    <bean id="incidentservice" class="org.apache.camel.example.axis.ReportIncidentService"/>

Running the unit test with mvn test reveals that it works nicely.

And since we use the @EndpointInjected that refers to the endpoint with the id backup directly we can loose the template tag in the xml, so its shorter:

    <bean id="incidentservice" class="org.apache.camel.example.axis.ReportIncidentService"/>

    <camel:camelContext id="camelContext">
        <!-- producer template exposed with this id -->
        <camel:template id="camelTemplate"/>

        <!-- endpoint named backup that is configued as a file component -->
        <camel:endpoint id="backup" uri="file://target?append=false"/>

    </camel:camelContext>

And the final touch we can do is that since the endpoint is injected with concrete endpoint to use we can remove the "backup" name parameter when we send the message. So we change from:

        // send the data to the endpoint and the header contains what filename it should be stored as
        template.sendBodyAndHeader("backup", data, "org.apache.camel.file.name", filename);

To without the name:

        // send the data to the endpoint and the header contains what filename it should be stored as
        template.sendBodyAndHeader(data, "org.apache.camel.file.name", filename);

Then we avoid to duplicate the name and if we rename the endpoint name then we don't forget to change it in the code also.

The End

This tutorial hasn't really touched the one of the key concept of Camel as a powerful routing and mediation framework. But we wanted to demonstrate its flexibility and that it integrates well with even older frameworks such as Apache Axis 1.4.

Check out the other tutorials on Camel and the other examples.

Note that the code shown here also applies to Camel 1.4 so actually you can get started right away with the released version of Camel. As this time of writing Camel 1.5 is work in progress.

See Also

Tutorial on using Camel in a Web Application

Camel has been designed to work great with the Spring framework; so if you are already a Spring user you can think of Camel as just a framework for adding to your Spring XML files.

So you can follow the usual Spring approach to working with web applications; namely to add the standard Spring hook to load a /WEB-INF/applicationContext.xml file. In that file you can include your usual Camel XML configuration.

Step1: Edit your web.xml

To enable spring add a context loader listener to your /WEB-INF/web.xml file

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">

  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>

</web-app>

This will cause Spring to boot up and look for the /WEB-INF/applicationContext.xml file.

Step 2: Create a /WEB-INF/applicationContext.xml file

Now you just need to create your Spring XML file and add your camel routes or configuration.

For example

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans 
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/context 
       http://www.springframework.org/schema/context/spring-context-2.5.xsd
       http://camel.apache.org/schema/spring 
       http://camel.apache.org/schema/spring/camel-spring.xsd">

  <camelContext xmlns="http://camel.apache.org/schema/spring">
    <route>
      <from uri="seda:foo"/>
      <to uri="mock:results"/>
    </route>
  </camelContext>

</beans>

Then boot up your web application and you're good to go!

Hints and Tips

If you use Maven to build your application your directory tree will look like this...

src/main/webapp/WEB-INF
  web.xml
  applicationContext.xml

You should update your Maven pom.xml to enable WAR packaging/naming like this...

<project>
    ...
    <packaging>war</packaging>
    ...
    <build>
	<finalName>[desired WAR file name]</finalName>
        ...
    </build>

To enable more rapid development we highly recommend the jetty:run maven plugin.

Please refer to the help for more information on using jetty:run - but briefly if you add the following to your pom.xml

  <build>
    <plugins>
      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>maven-jetty-plugin</artifactId>
        <configuration>
          <webAppConfig>
            <contextPath>/</contextPath>
          </webAppConfig>
          <scanIntervalSeconds>10</scanIntervalSeconds>
        </configuration>
      </plugin>
    </plugins>
  </build>

Then you can run your web application as follows

mvn jetty:run

Then Jetty will also monitor your target/classes directory and your src/main/webapp directory so that if you modify your spring XML, your web.xml or your java code the web application will be restarted, re-creating your Camel routes.

If your unit tests take a while to run, you could miss them out when running your web application via

mvn -Dtest=false jetty:run

Tutorial Business Partners

Under Construction

This tutorial is a work in progress.

Background and Introduction

Business Background

So there's a company, which we'll call Acme. Acme sells widgets, in a fairly unusual way. Their customers are responsible for telling Acme what they purchased. The customer enters into their own systems (ERP or whatever) which widgets they bought from Acme. Then at some point, their systems emit a record of the sale which needs to go to Acme so Acme can bill them for it. Obviously, everyone wants this to be as automated as possible, so there needs to be integration between the customer's system and Acme.

Sadly, Acme's sales people are, technically speaking, doormats. They tell all their prospects, "you can send us the data in whatever format, using whatever protocols, whatever. You just can't change once it's up and running."

The result is pretty much what you'd expect. Taking a random sample of 3 customers:

  • Customer 1: XML over FTP
  • Customer 2: CSV over HTTP
  • Customer 3: Excel via e-mail

Now on the Acme side, all this has to be converted to a canonical XML format and submitted to the Acme accounting system via JMS. Then the Acme accounting system does its stuff and sends an XML reply via JMS, with a summary of what it processed (e.g. 3 line items accepted, line item #2 in error, total invoice $123.45). Finally, that data needs to be formatted into an e-mail, and sent to a contact at the customer in question ("Dear Joyce, we received an invoice on 1/2/08. We accepted 3 line items totaling $123.45, though there was an error with line items #2 [invalid quantity ordered]. Thank you for your business. Love, Acme.").

So it turns out Camel can handle all this:

  • Listen for HTTP, e-mail, and FTP files
  • Grab attachments from the e-mail messages
  • Convert XML, XLS, and CSV files to a canonical XML format
  • read and write JMS messages
  • route based on company ID
  • format e-mails using Velocity templates
  • send outgoing e-mail messages

Tutorial Background

This tutorial will cover all that, plus setting up tests along the way.

Before starting, you should be familiar with:

You'll learn:

  • How to set up a Maven build for a Camel project
  • How to transform XML, CSV, and Excel data into a standard XML format with Camel
    • How to write POJOs (Plain Old Java Objects), Velocity templates, and XSLT stylesheets that are invoked by Camel routes for message transformation
  • How to configure simple and complex Routes in Camel, using either the XML or the Java DSL format
  • How to set up unit tests that load a Camel configuration and test Camel routes
  • How to use Camel's Data Formats to automatically convert data between Java objects and XML, CSV files, etc.
  • How to send and receive e-mail from Camel
  • How to send and receive JMS messages from Camel
  • How to use Enterprise Integration Patterns including Message Router and Pipes and Filters
    • How to use various languages to express content-based routing rules in Camel
  • How to deal with Camel messages, headers, and attachments

You may choose to treat this as a hands-on tutorial, and work through building the code and configuration files yourself. Each of the sections gives detailed descriptions of the steps that need to be taken to get the components and routes working in Camel, and takes you through tests to make sure they are working as expected.

But each section also links to working copies of the source and configuration files, so if you don't want the hands-on approach, you can simply review and/or download the finished files.

High-Level Diagram

Here's more or less what the integration process looks like.

First, the input from the customers to Acme:

And then, the output from Acme to the customers:

Tutorial Tasks

To get through this scenario, we're going to break it down into smaller pieces, implement and test those, and then try to assemble the big scenario and test that.

Here's what we'll try to accomplish:

  1. Create a Maven build for the project
  2. Get sample files for the customer Excel, CSV, and XML input
  3. Get a sample file for the canonical XML format that Acme's accounting system uses
  4. Create an XSD for the canonical XML format
  5. Create JAXB POJOs corresponding to the canonical XSD
  6. Create an XSLT stylesheet to convert the Customer 1 (XML over FTP) messages to the canonical format
  7. Create a unit test to ensure that a simple Camel route invoking the XSLT stylesheet works
  8. Create a POJO that converts a List<List<String>> to the above JAXB POJOs
    • Note that Camel can automatically convert CSV input to a List of Lists of Strings representing the rows and columns of the CSV, so we'll use this POJO to handle Customer 2 (CSV over HTTP)
  9. Create a unit test to ensure that a simple Camel route invoking the CSV processing works
  10. Create a POJO that converts a Customer 3 Excel file to the above JAXB POJOs (using POI to read Excel)
  11. Create a unit test to ensure that a simple Camel route invoking the Excel processing works
  12. Create a POJO that reads an input message, takes an attachment off the message, and replaces the body of the message with the attachment
    • This is assuming for Customer 3 (Excel over e-mail) that the e-mail contains a single Excel file as an attachment, and the actual e-mail body is throwaway
  13. Build a set of Camel routes to handle the entire input (Customer -> Acme) side of the scenario.
  14. Build unit tests for the Camel input.
  15. TODO: Tasks for the output (Acme -> Customer) side of the scenario

Let's Get Started!

Step 1: Initial Maven build

We'll use Maven for this project as there will eventually be quite a few dependencies and it's nice to have Maven handle them for us. You should have a current version of Maven (e.g. 2.0.9) installed.

You can start with a pretty empty project directory and a Maven POM file, or use a simple JAR archetype to create one.

Here's a sample POM. We've added a dependency on camel-core, and set the compile version to 1.5 (so we can use annotations):

pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.camel.tutorial</groupId>
    <artifactId>business-partners</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>Camel Business Partners Tutorial</name>
    <dependencies>
        <dependency>
            <artifactId>camel-core</artifactId>
            <groupId>org.apache.camel</groupId>
            <version>1.4.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Step 2: Get Sample Files

You can make up your own if you like, but here are the "off the shelf" ones. You can save yourself some time by downloading these to src/test/resources in your Maven project.

If you look at these files, you'll see that the different input formats use different field names and/or ordering, because of course the sales guys were totally OK with that. Sigh.

Step 3: XSD and JAXB Beans for the Canonical XML Format

Here's the sample of the canonical XML file:

<?xml version="1.0" encoding="UTF-8"?>
<invoice xmlns="http://activemq.apache.org/camel/tutorial/partners/invoice">
  <partner-id>2</partner-id>
  <date-received>9/12/2008</date-received>
  <line-item>
    <product-id>134</product-id>
    <description>A widget</description>
    <quantity>3</quantity>
    <item-price>10.45</item-price>
    <order-date>6/5/2008</order-date>
  </line-item>
  <!-- // more line-item elements here -->
  <order-total>218.82</order-total>
</invoice>

If you're ambitions, you can write your own XSD (XML Schema) for files that look like this, and save it to src/main/xsd.

Solution: If not, you can download mine, and save that to save it to src/main/xsd.

Generating JAXB Beans

Down the road we'll want to deal with the XML as Java POJOs. We'll take a moment now to set up those XML binding POJOs. So we'll update the Maven POM to generate JAXB beans from the XSD file.

We need a dependency:

<dependency>
    <artifactId>camel-jaxb</artifactId>
    <groupId>org.apache.camel</groupId>
    <version>1.4.0</version>
</dependency>

And a plugin configured:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>jaxb2-maven-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>xjc</goal>
            </goals>
        </execution>
    </executions>
</plugin>

That should do it (it automatically looks for XML Schemas in src/main/xsd to generate beans for). Run mvn install and it should emit the beans into target/generated-sources/jaxb. Your IDE should see them there, though you may need to update the project to reflect the new settings in the Maven POM.

Step 4: Initial Work on Customer 1 Input (XML over FTP)

To get a start on Customer 1, we'll create an XSLT template to convert the Customer 1 sample file into the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the XSLT template is valid and can be run safely in Camel.

Create an XSLT template

Start with the Customer 1 sample input. You want to create an XSLT template to generate XML like the canonical XML sample above – an invoice element with line-item elements (one per item in the original XML document). If you're especially clever, you can populate the current date and order total elements too.

Solution: My sample XSLT template isn't that smart, but it'll get you going if you don't want to write one of your own.

Create a unit test

Here's where we get to some meaty Camel work. We need to:

  • Set up a unit test
  • That loads a Camel configuration
  • That has a route invoking our XSLT
  • Where the test sends a message to the route
  • And ensures that some XML comes out the end of the route

The easiest way to do this is to set up a Spring context that defines the Camel stuff, and then use a base unit test class from Spring that knows how to load a Spring context to run tests against. So, the procedure is:

Set Up a Skeletal Camel/Spring Unit Test
  1. Add dependencies on Camel-Spring, and the Spring test JAR (which will automatically bring in JUnit 3.8.x) to your POM:
    <dependency>
        <artifactId>camel-spring</artifactId>
        <groupId>org.apache.camel</groupId>
        <version>1.4.0</version>
    </dependency>
    <dependency>
        <artifactId>spring-test</artifactId>
        <groupId>org.springframework</groupId>
        <version>2.5.5</version>
        <scope>test</scope>
    </dependency>
    
  2. Create a new unit test class in src/test/java/your-package-here, perhaps called XMLInputTest.java
  3. Make the test extend Spring's AbstractJUnit38SpringContextTests class, so it can load a Spring context for the test
  4. Create a Spring context configuration file in src/test/resources, perhaps called XMLInputTest-context.xml
  5. In the unit test class, use the class-level @ContextConfiguration annotation to indicate that a Spring context should be loaded
    • By default, this looks for a Context configuration file called TestClassName-context.xml in a subdirectory corresponding to the package of the test class. For instance, if your test class was org.apache.camel.tutorial.XMLInputTest, it would look for org/apache/camel/tutorial/XMLInputTest-context.xml
    • To override this default, use the locations attribute on the @ContextConfiguration annotation to provide specific context file locations (starting each path with a / if you don't want it to be relative to the package directory). My solution does this so I can put the context file directly in src/test/resources instead of in a package directory under there.
  6. Add a CamelContext instance variable to the test class, with the @Autowired annotation. That way Spring will automatically pull the CamelContext out of the Spring context and inject it into our test class.
  7. Add a ProducerTemplate instance variable and a setUp method that instantiates it from the CamelContext. We'll use the ProducerTemplate later to send messages to the route.
    protected ProducerTemplate<Exchange> template;
    
    protected void setUp() throws Exception {
        super.setUp();
        template = camelContext.createProducerTemplate();
    }
    
  8. Put in an empty test method just for the moment (so when we run this we can see that "1 test succeeded")
  9. Add the Spring <beans> element (including the Camel Namespace) with an empty <camelContext> element to the Spring context, like this:
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.springframework.org/schema/beans
                                   http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                               http://activemq.apache.org/camel/schema/spring
                                   http://activemq.apache.org/camel/schema/spring/camel-spring-1.4.0.xsd">
    
        <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">
        </camelContext>
    </beans>
    

Test it by running mvn install and make sure there are no build errors. So far it doesn't test much; just that your project and test and source files are all organized correctly, and the one empty test method completes successfully.

Solution: Your test class might look something like this:

  • src/test/java/org/apache/camel/tutorial/XMLInputTest.java
  • src/test/resources/XMLInputTest-context.xml (same as just above)
Flesh Out the Unit Test

So now we're going to write a Camel route that applies the XSLT to the sample Customer 1 input file, and makes sure that some XML output comes out:

  1. Save the input-customer1.xml file to src/test/resources
  2. Save your XSLT file (created in the previous step) to src/main/resources
  3. Write a Camel Route, either right in the Spring XML, or using the Java DSL (in another class under src/test/java somewhere). This route should use the Pipes and Filters integration pattern to:
    1. Start from the endpoint direct:start (which lets the test conveniently pass messages into the route)
    2. Call the endpoint xslt:YourXSLTFile.xsl (to transform the message with the specified XSLT template)
    3. Send the result to the endpoint mock:finish (which lets the test verify the route output)
  4. Add a test method to the unit test class that:
    1. Get a reference to the Mock endpoint mock:finish using code like this:
      MockEndpoint finish = MockEndpoint.resolve(camelContext, "mock:finish");
      
    2. Set the expectedMessageCount on that endpoint to 1
    3. Get a reference to the Customer 1 input file, using code like this:
      InputStream in = XMLInputTest.class.getResourceAsStream("/input-partner1.xml");
      assertNotNull(in);
      
    4. Send that InputStream as a message to the direct:start endpoint, using code like this:
          template.sendBody("direct:start", in);
      
      Note that we can send the sample file body in several formats (File, InputStream, String, etc.) but in this case an InputStream is pretty convenient.
    5. Ensure that the message made it through the route to the final endpoint, by testing all configured Mock endpoints like this:
      MockEndpoint.assertIsSatisfied(camelContext);
      
    6. If you like, inspect the final message body using some code like finish.getExchanges().get(0).getIn().getBody().
      • If you do this, you'll need to know what format that body is – String, byte array, InputStream, etc.
  5. Run your test with mvn install and make sure the build completes successfully.

Solution: Your finished test might look something like this:

Test Base Class

Once your test class is working, you might want to extract things like the @Autowired CamelContext, the ProducerTemplate, and the setUp method to a custom base class that you extend with your other tests.

Step 5: Initial Work on Customer 2 Input (CSV over HTTP)

To get a start on Customer 2, we'll create a POJO to convert the Customer 2 sample CSV data into the JAXB POJOs representing the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the CSV conversion and JAXB handling is valid and can be run safely in Camel.

Create a CSV-handling POJO

To begin with, CSV is a known data format in Camel. Camel can convert a CSV file to a List (representing rows in the CSV) of Lists (representing cells in the row) of Strings (the data for each cell). That means our POJO can just assume the data coming in is of type List<List<String>>, and we can declare a method with that as the argument.

Looking at the JAXB code in target/generated-sources/jaxb, it looks like an Invoice object represents the whole document, with a nested list of LineItemType objects for the line items. Therefore our POJO method will return an Invoice (a document in the canonical XML format).

So to implement the CSV-to-JAXB POJO, we need to do something like this:

  1. Create a new class under src/main/java, perhaps called CSVConverterBean.
  2. Add a method, with one argument of type List<List<String>> and the return type Invoice
    • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
  3. In the method, the logic should look roughly like this:
    1. Create a new Invoice, using the method on the generated ObjectFactory class
    2. Loop through all the rows in the incoming CSV (the outer List)
    3. Skip the first row, which contains headers (column names)
    4. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values (the Strings in the inner List) and put them into the correct fields of the LineItemType
        • Not all of the values will actually go into the line item in this example
        • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
        • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means using a SimpleDateFormat to parse the date and setting that date on a GregorianCalendar
      3. Add the line item to the invoice
    5. Populate the partner ID, date of receipt, and order total on the Invoice
    6. Throw any exceptions out of the method, so Camel knows something went wrong
    7. Return the finished Invoice

Solution: Here's an example of what the CSVConverterBean might look like.

Create a unit test

Start with a simple test class and test Spring context like last time, perhaps based on the name CSVInputTest:

CSVInputTest.java
/**
 * A test class the ensure we can convert Partner 2 CSV input files to the
 * canonical XML output format, using JAXB POJOs.
 */
@ContextConfiguration(locations = "/CSVInputTest-context.xml")
public class CSVInputTest extends AbstractJUnit38SpringContextTests {
    @Autowired
    protected CamelContext camelContext;
    protected ProducerTemplate<Exchange> template;

    protected void setUp() throws Exception {
        super.setUp();
        template = camelContext.createProducerTemplate();
    }

    public void testCSVConversion() {
        // TODO
    }
}
CSVInputTest-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                               http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://activemq.apache.org/camel/schema/spring
                               http://activemq.apache.org/camel/schema/spring/camel-spring-1.4.0.xsd">

    <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">
        <!-- TODO -->
    </camelContext>
</beans>

Now the meaty part is to flesh out the test class and write the Camel routes.

  1. Update the Maven POM to include CSV Data Format support:
    <dependency>
        <artifactId>camel-csv</artifactId>
        <groupId>org.apache.camel</groupId>
        <version>1.4.0</version>
    </dependency>
    
  2. Write the routes (right in the Spring XML context, or using the Java DSL) for the CSV conversion process, again using the Pipes and Filters pattern:
    1. Start from the endpoint direct:CSVstart (which lets the test conveniently pass messages into the route). We'll name this differently than the starting point for the previous test, in case you use the Java DSL and put all your routes in the same package (which would mean that each test would load the DSL routes for several tests.)
    2. This time, there's a little preparation to be done. Camel doesn't know that the initial input is a CSV, so it won't be able to convert it to the expected List<List<String>> without a little hint. For that, we need an unmarshal transformation in the route. The unmarshal method (in the DSL) or element (in the XML) takes a child indicating the format to unmarshal; in this case that should be csv.
    3. Next invoke the POJO to transform the message with a bean:CSVConverter endpoint
    4. As before, send the result to the endpoint mock:finish (which lets the test verify the route output)
    5. Finally, we need a Spring <bean> element in the Spring context XML file (but outside the <camelContext> element) to define the Spring bean that our route invokes. This Spring bean should have a name attribute that matches the name used in the bean endpoint (CSVConverter in the example above), and a class attribute that points to the CSV-to-JAXB POJO class you wrote above (such as, org.apache.camel.tutorial.CSVConverterBean). When Spring is in the picture, any bean endpoints look up Spring beans with the specified name.
  3. Write a test method in the test class, which should look very similar to the previous test class:
    1. Get the MockEndpoint for the final endpoint, and tell it to expect one message
    2. Load the Partner 2 sample CSV file from the ClassPath, and send it as the body of a message to the starting endpoint
    3. Verify that the final MockEndpoint is satisfied (that is, it received one message) and examine the message body if you like
      • Note that we didn't marshal the JAXB POJOs to XML in this test, so the final message should contain an Invoice as the body. You could write a simple line of code to get the Exchange (and Message) from the MockEndpoint to confirm that.
  4. Run this new test with mvn install and make sure it passes and the build completes successfully.

Solution: Your finished test might look something like this:

Step 6: Initial Work on Customer 3 Input (Excel over e-mail)

To get a start on Customer 3, we'll create a POJO to convert the Customer 3 sample Excel data into the JAXB POJOs representing the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the Excel conversion and JAXB handling is valid and can be run safely in Camel.

Create an Excel-handling POJO

Camel does not have a data format handler for Excel by default. We have two options – create an Excel DataFormat (so Camel can convert Excel spreadsheets to something like the CSV List<List<String>> automatically), or create a POJO that can translate Excel data manually. For now, the second approach is easier (if we go the DataFormat route, we need code to both read and write Excel files, whereas otherwise read-only will do).

So, we need a POJO with a method that takes something like an InputStream or byte[] as an argument, and returns in Invoice as before. The process should look something like this:

  1. Update the Maven POM to include POI support:
    <dependency>
        <artifactId>poi</artifactId>
        <groupId>org.apache.poi</groupId>
        <version>3.1-FINAL</version>
    </dependency>
    
  2. Create a new class under src/main/java, perhaps called ExcelConverterBean.
  3. Add a method, with one argument of type InputStream and the return type Invoice
    • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
  4. In the method, the logic should look roughly like this:
    1. Create a new Invoice, using the method on the generated ObjectFactory class
    2. Create a new HSSFWorkbook from the InputStream, and get the first sheet from it
    3. Loop through all the rows in the sheet
    4. Skip the first row, which contains headers (column names)
    5. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values and put them into the correct fields of the LineItemType (you'll need some data type conversion logic)
        • Not all of the values will actually go into the line item in this example
        • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
        • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means setting the date from a date cell on a GregorianCalendar
      3. Add the line item to the invoice
    6. Populate the partner ID, date of receipt, and order total on the Invoice
    7. Throw any exceptions out of the method, so Camel knows something went wrong
    8. Return the finished Invoice

Solution: Here's an example of what the ExcelConverterBean might look like.

Create a unit test

The unit tests should be pretty familiar now. The test class and context for the Excel bean should be quite similar to the CSV bean.

  1. Create the basic test class and corresponding Spring Context XML configuration file
  2. The XML config should look a lot like the CSV test, except:
    • Remember to use a different start endpoint name if you're using the Java DSL and not use separate packages per test
    • You don't need the unmarshal step since the Excel POJO takes the raw InputStream from the source endpoint
    • You'll declare a <bean> and endpoint for the Excel bean prepared above instead of the CSV bean
  3. The test class should look a lot like the CSV test, except use the right input file name and start endpoint name.

Logging

You may notice that your tests emit a lot less output all of a sudden. The dependency on POI brought in Log4J and configured commons-logging to use it, so now we need a log4j.properties file to configure log output. You can use the attached one (snarfed from ActiveMQ) or write your own; either way save it to src/main/resources to ensure you continue to see log output.

Solution: Your finished test might look something like this:

Step 7: Put this all together into Camel routes for the Customer Input

With all the data type conversions working, the next step is to write the real routes that listen for HTTP, FTP, or e-mail input, and write the final XML output to an ActiveMQ queue. Along the way these routes will use the data conversions we've developed above.

So we'll create 3 routes to start with, as shown in the diagram back at the beginning:

  1. Accept XML orders over FTP from Customer 1 (we'll assume the FTP server dumps files in a local directory on the Camel machine)
  2. Accept CSV orders over HTTP from Customer 2
  3. Accept Excel orders via e-mail from Customer 3 (we'll assume the messages are sent to an account we can access via IMAP)

...

Step 8: Create a unit test for the Customer Input Routes

Languages Supported Appendix

To support flexible and powerful Enterprise Integration Patterns Camel supports various Languages to create an Expression or Predicate within either the Routing Domain Specific Language or the Xml Configuration. The following languages are supported

Bean Language

The purpose of the Bean Language is to be able to implement an Expression or Predicate using a simple method on a bean. The bean name is resolved using a Registry, such as the Spring ApplicationContext, then a method is invoked to evaluate the Expression or Predicate. If no method name is provided then one is chosen using the rules for Bean Binding; using the type of the message body and using any annotations on the bean methods.

The Bean Binding rules are used to bind the Message Exchange to the method parameters; so you can annotate the bean to extract headers or other expressions such as XPath or XQuery from the message.

Using Bean Expressions in Java

from("activemq:topic:OrdersTopic")
  .filter().method("myBean", "isGoldCustomer")
    .to("activemq:BigSpendersQueue");

Using Bean Expressions in Spring XML

<route>
  <from uri="activemq:topic:OrdersTopic"/>
  <filter>
    <method ref="myBean" method="isGoldCustomer"/>
    <to uri="activemq:BigSpendersQueue"/>
  </filter>
</route>

Bean Attribute Now Deprecated

The bean attribute of the method expression element is now deprecated. Use the ref attribute instead.

Writing the Expression Bean

The bean in the above examples is just any old Java Bean with a method called isGoldCustomer() that returns some object that is easily converted to a boolean value in this case, as its used as a predicate.

Example:

public class MyBean {
  public boolean isGoldCustomer(Exchange exchange) {
  	 // ...
  }
}

We can also use the Bean Integration annotations.

Example:

public boolean isGoldCustomer(String body) {...}

or

public boolean isGoldCustomer(@Header(name = "foo") Integer fooHeader) {...}

So you can bind parameters of the method to the Exchange, the Message or individual headers, properties, the body or other expressions.

Non-Registry Beans

The Bean Language also supports invoking beans that isn't registered in the Registry. This is usable for quickly to invoke a bean from Java DSL where you don't need to register the bean in the Registry such as the Spring ApplicationContext. Camel can instantiate the bean and invoke the method if given a class or invoke an already existing instance.

Example:

from("activemq:topic:OrdersTopic")
  .filter().expression(BeanLanguage(MyBean.class, "isGoldCustomer"))
  .to("activemq:BigSpendersQueue");

The 2nd parameter isGoldCustomer is an optional parameter to explicit set the method name to invoke. If not provided Camel will try to invoke the most suitable method. If case of ambiguity Camel will thrown an Exception. In these situations the 2nd parameter can solve this problem. Also the code is more readable if the method name is provided. The 1st parameter can also be an existing instance of a Bean such as:

private MyBean my;

from("activemq:topic:OrdersTopic")
  .filter().expression(BeanLanguage.bean(my, "isGoldCustomer"))
  .to("activemq:BigSpendersQueue");

In Camel 2.2: you can avoid the BeanLanguage and have it just as:

private MyBean my;

from("activemq:topic:OrdersTopic")
  .filter().expression(bean(my, "isGoldCustomer"))
  .to("activemq:BigSpendersQueue");

Which also can be done in a bit shorter and nice way:

private MyBean my;

from("activemq:topic:OrdersTopic")
  .filter().method(my, "isGoldCustomer")
  .to("activemq:BigSpendersQueue");

Other Examples

We have some test cases you can look at if it'll help

  • MethodFilterTest is a JUnit test case showing the Java DSL use of the bean expression being used in a filter
  • aggregator.xml is a Spring XML test case for the Aggregator which uses a bean method call to test for the completion of the aggregation.

Dependencies

The Bean language is part of camel-core.

Constant Expression Language

The Constant Expression Language is really just a way to specify constant strings as a type of expression.

Example usage

The setHeader element of the Spring DSL can utilize a constant expression like:

<route>
  <from uri="seda:a"/>
  <setHeader headerName="theHeader">
    <constant>the value</constant>        
  </setHeader>
  <to uri="mock:b"/>     
</route>

In this case, the Message coming from the seda:a Endpoint will have theHeader header set to the constant value the value.

And the same example using Java DSL:

from("seda:a")
  .setHeader("theHeader", constant("the value"))
  .to("mock:b");

Dependencies

The Constant language is part of camel-core.

EL

Camel supports the unified JSP and JSF Expression Language via the JUEL to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

For example you could use EL inside a Message Filter in XML

<route>
  <from uri="seda:foo"/>
  <filter>
    <el>${in.headers.foo == 'bar'}</el>
    <to uri="seda:bar"/>
  </filter>
</route>

You could also use slightly different syntax, e.g. if the header name is not a valid identifier:

<route>
  <from uri="seda:foo"/>
  <filter>
    <el>${in.headers['My Header'] == 'bar'}</el>
    <to uri="seda:bar"/>
  </filter>
</route>

You could use EL to create an Predicate in a Message Filter or as an Expression for a Recipient List

Variables

Variable

Type

Description

exchange

Exchange

the Exchange object

in

Message

the exchange.in message

out

Message

the exchange.out message

Samples

You can use EL dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamiliyName method then you can construct the syntax as follows:

"${in.body.familyName}"

Dependencies

To use EL in your camel routes you need to add the a dependency on camel-juel which implements the EL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-juel</artifactId>
  <version>x.x.x</version>
</dependency>

Otherwise you'll also need to include JUEL.

Header Expression Language

The Header Expression Language allows you to extract values of named headers.

Example usage

The recipientList element of the Spring DSL can utilize a header expression like:

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/recipientListWithStringDelimitedHeader.xml}

In this case, the list of recipients are contained in the header 'myHeader'.

And the same example in Java DSL:

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RecipientListWithStringDelimitedHeaderTest.java}

And with a slightly different syntax where you use the builder to the fullest (i.e. avoid using parameters but using stacked operations, notice that header is not a parameter but a stacked method call)

java from("direct:a").recipientList().header("myHeader");

Dependencies

The Header language is part of camel-core.

Error rendering macro 'include'

org.owasp.validator.html.ScanException: java.util.EmptyStackException

Mvel

Camel allows Mvel to be used as an Expression or Predicate the DSL or Xml Configuration.

You could use Mvel to create an Predicate in a Message Filter or as an Expression for a Recipient List

You can use Mvel dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamiliyName method then you can construct the syntax as follows:

"request.body.familyName"
   // or 
"getRequest().getBody().getFamilyName()"

Variables

Variable

Type

Description

this

Exchange

the Exchange is the root object

exchange

Exchange

the Exchange object

exception

Throwable

the Exchange exception (if any)

exchangeId

String

the exchange id

fault

Message

the Fault message (if any)

request

Message

the exchange.in message

response

Message

the exchange.out message (if any)

properties

Map

the exchange properties

property(name)

Object

the property by the given name

property(name, type)

Type

the property by the given name as the given type

Samples

For example you could use Mvel inside a Message Filter in XML

<route>
  <from uri="seda:foo"/>
  <filter>
    <mvel>request.headers.foo == 'bar'</mvel>
    <to uri="seda:bar"/>
  </filter>
</route>

And the sample using Java DSL:

   from("seda:foo").filter().mvel("request.headers.foo == 'bar'").to("seda:bar");

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

.setHeader("myHeader").mvel("resource:classpath:script.mvel")

Dependencies

To use Mvel in your camel routes you need to add the a dependency on camel-mvel which implements the Mvel language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-mvel</artifactId>
  <version>x.x.x</version>
</dependency>

OGNL

Camel allows OGNL to be used as an Expression or Predicate the DSL or Xml Configuration.

You could use OGNL to create an Predicate in a Message Filter or as an Expression for a Recipient List

You can use OGNL dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamilyName method then you can construct the syntax as follows:

"request.body.familyName"
   // or 
"getRequest().getBody().getFamilyName()"

Variables

Variable

Type

Description

this

Exchange

the Exchange is the root object

exchange

Exchange

the Exchange object

exception

Throwable

the Exchange exception (if any)

exchangeId

String

the exchange id

fault

Message

the Fault message (if any)

request

Message

the exchange.in message

response

Message

the exchange.out message (if any)

properties

Map

the exchange properties

property(name)

Object

the property by the given name

property(name, type)

Type

the property by the given name as the given type

Samples

For example you could use OGNL inside a Message Filter in XML

<route>
  <from uri="seda:foo"/>
  <filter>
    <ognl>request.headers.foo == 'bar'</ognl>
    <to uri="seda:bar"/>
  </filter>
</route>

And the sample using Java DSL:

   from("seda:foo").filter().ognl("request.headers.foo == 'bar'").to("seda:bar");

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

.setHeader("myHeader").ognl("resource:classpath:myognl.txt")

Dependencies

To use OGNL in your camel routes you need to add the a dependency on camel-ognl which implements the OGNL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-ognl</artifactId>
  <version>x.x.x</version>
</dependency>

Otherwise, you'll also need OGNL

Property Expression Language

The Property Expression Language allows you to extract values of named exchange properties.

From Camel 2.15 onwards the property language has been renamed to exchangeProperty to avoid ambiguity, confusion and clash with properties as a general term. So use exchangeProperty instead of property when using Camel 2.15 onwards.

 

Example usage

The recipientList element of the Spring DSL can utilize a property expression like:

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/recipientListWithStringDelimitedProperty.xml}

In this case, the list of recipients are contained in the property 'myProperty'.

And the same example in Java DSL:

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RecipientListWithStringDelimitedPropertyTest.java}

And with a slightly different syntax where you use the builder to the fullest (i.e. avoid using parameters but using stacked operations, notice that property is not a parameter but a stacked method call)

java from("direct:a").recipientList().property("myProperty");

Dependencies

The Property language is part of camel-core.

Scripting Languages

Camel supports a number of scripting languages which can be used to create an Expression or Predicate via the standard JSR 223 which is a standard part of Java 6.

The following scripting languages are integrated into the DSL:

Language

DSL keyword

EL

el

Groovy

groovy

JavaScript

javaScript

JoSQL

sql

JXPath

jxpath

MVEL

mvel

OGNL

ognl

PHP

php

Python

python

Ruby

ruby

XPath

xpath

XQuery

xquery

However any JSR 223 scripting language can be used using the generic DSL methods.

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

See Also

BeanShell

Camel supports BeanShell among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a BeanShell expression use the following Java code:

...choice()
    .when(script("beanshell", "request.getHeaders().get(\"foo\").equals(\"bar\")"))
       .to("...")

Or the something like this in your Spring XML:

<filter>
  <language language="beanshell">request.getHeaders().get("Foo") == null</language>
  ...

BeanShell Issues

You must use BeanShell 2.0b5 or greater. Note that as of 2.0b5 BeanShell cannot compile scripts, which causes Camel releases before 2.6 to fail when configured with BeanShell expressions.

You could follow the examples above to create an Predicate in a Message Filter or as an Expression for a Recipient List

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

JavaScript

Camel supports JavaScript/ECMAScript among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a JavaScript expression use the following Java code

... javaScript("someJavaScriptExpression") ... 

For example you could use the javaScript function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use JavaScript to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

    from("direct:start")
        .choice()
            .when().javaScript("request.headers.get('user') == 'admin'").to("seda:adminQueue")
        .otherwise()
            .to("seda:regularQueue");

And a Spring DSL sample as well:

    <route>
        <from uri="direct:start"/>
        <choice>
            <when>
                <javaScript>request.headers.get('user') == 'admin'</javaScript>
                <to uri="seda:adminQueue"/>
            </when>
            <otherwise>
                <to uri="seda:regularQueue"/>
            </otherwise>
        </choice>
    </route>

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

Groovy

Camel supports Groovy among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Groovy expression use the following Java code

... groovy("someGroovyExpression") ... 

For example you could use the groovy function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Dependency

You should add the camel-groovy dependeny when using Groovy language with Camel. The generic camel-script is not optimized for best Groovy experience, and hence you should add camel-groovy as dependency.

Customizing Groovy Shell

Sometimes you may need to use custom GroovyShell instance in your Groovy expressions. To provide custom GroovyShell, add implementation of the org.apache.camel.language.groovy.GroovyShellFactory SPI interface to your Camel registry. For example after adding the following bean to your Spring context...

public class CustomGroovyShellFactory implements GroovyShellFactory {
 
  public GroovyShell createGroovyShell(Exchange exchange) {
    ImportCustomizer importCustomizer = new ImportCustomizer();
    importCustomizer.addStaticStars("com.example.Utils");
    CompilerConfiguration configuration = new CompilerConfiguration();
    configuration.addCompilationCustomizers(importCustomizer);
    return new GroovyShell(configuration);
  }

}

...Camel will use your custom GroovyShell instance (containing your custom static imports), instead of the default one.

Example

// lets route if a line item is over $100
from("queue:foo").filter(groovy("request.lineItems.any { i -> i.value > 100 }")).to("queue:bar")

And the Spring DSL:

        <route>
            <from uri="queue:foo"/>
            <filter>
                <groovy>request.lineItems.any { i -> i.value > 100 }</groovy>
                <to uri="queue:bar"/>
            </filter>
        </route>

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

Python

Camel supports Python among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Python expression use the following Java code

... python("somePythonExpression") ... 

For example you could use the python function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use Python to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

    from("direct:start")
        .choice()
            .when().python("request.headers['user'] == 'admin'").to("seda:adminQueue")
        .otherwise()
            .to("seda:regularQueue");

And a Spring DSL sample as well:

    <route>
        <from uri="direct:start"/>
        <choice>
            <when>
                <python>request.headers['user'] == 'admin'</python>
                <to uri="seda:adminQueue"/>
            </when>
            <otherwise>
                <to uri="seda:regularQueue"/>
            </otherwise>
        </choice>
    </route>

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

PHP

Camel supports PHP among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a PHP expression use the following Java code

... php("somePHPExpression") ... 

For example you could use the php function to create an Predicate in a Message Filter or as an Expression for a Recipient List

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

Ruby

Camel supports Ruby among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Ruby expression use the following Java code

... ruby("someRubyExpression") ... 

For example you could use the ruby function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use Ruby to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

    from("direct:start")
        .choice()
            .when().ruby("$request.headers['user'] == 'admin'").to("seda:adminQueue")
        .otherwise()
            .to("seda:regularQueue");

And a Spring DSL sample as well:

    <route>
        <from uri="direct:start"/>
        <choice>
            <when>
                <ruby>$request.headers['user'] == 'admin'</ruby>
                <to uri="seda:adminQueue"/>
            </when>
            <otherwise>
                <to uri="seda:regularQueue"/>
            </otherwise>
        </choice>
    </route>

ScriptContext Options

 

The JSR-223 scripting language's ScriptContext is pre-configured with the following attributes all set at ENGINE_SCOPE.

Attribute

Type

Value

camelContext

org.apache.camel.CamelContext

The Camel Context.

context

org.apache.camel.CamelContext

The Camel Context (cannot be used in groovy).

exchange

org.apache.camel.Exchange

The current Exchange.

properties

org.apache.camel.builder.script.PropertiesFunction

Camel 2.9: Function with a resolve method to make it easier to use Camels Properties component from scripts. See further below for example.

request

org.apache.camel.Message

The IN message.

response

org.apache.camel.Message

Deprecated: The OUT message. The OUT message is null by default. Use the IN message instead.

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

public void testArgumentsExample() throws Exception {
    getMockEndpoint("mock:result").expectedMessageCount(0);
    getMockEndpoint("mock:unmatched").expectedMessageCount(1);

    // additional arguments to ScriptEngine
    Map<String, Object> arguments = new HashMap<>();
    arguments.put("foo", "bar");
    arguments.put("baz", 7);

    // those additional arguments is provided as a header on the Camel Message
    template.sendBodyAndHeader("direct:start", "hello", ScriptBuilder.ARGUMENTS, arguments);

    assertMockEndpointsSatisfied();


 


Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

.setHeader("myHeader").groovy("context.resolvePropertyPlaceholders('{{' + request.headers.get('foo') + '}}')")

From Camel 2.9: you can now use the properties function and the same example is simpler:

.setHeader("myHeader").groovy("properties.resolve(request.headers.get('foo'))")


Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

.setHeader("myHeader").groovy("resource:classpath:mygroovy.groovy")


How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

textbar = "baz"; # some other statements ... # camel take the result value as the script evaluation result result = body * 2 + 1

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-script</artifactId>
 <version>x.x.x</version>
</dependency>

Simple Expression Language

The Simple Expression Language was a really simple language when it was created, but has since grown more powerful. It is primarily intended for being a really small and simple language for evaluating Expressions and Predicates without requiring any new dependencies or knowledge of XPath; so it is ideal for testing in camel-core. The idea was to cover 95% of the common use cases when you need a little bit of expression based script in your Camel routes.

However for much more complex use cases you are generally recommended to choose a more expressive and powerful language such as:

The simple language uses ${body} placeholders for complex expressions where the expression contains constant literals.

Deprecated: The ${} placeholders can be omitted if the expression starts with the token, or if the token is only itself.

Alternative syntax

From Camel 2.5 you can also use the alternative syntax which uses $simple{} as placeholders. This can be used in situations to avoid clashes when using for example Spring property placeholder together with Camel.

Configuring result type

From Camel 2.8 you can configure the result type of the Simple expression. For example to set the type as a java.lang.Boolean or a java.lang.Integer etc.

File language is now merged with Simple language

From Camel 2.2, the File Language is now merged with Simple language which means you can use all the file syntax directly within the simple language.

Simple Language Changes in Camel 2.9 onwards

The Simple language have been improved from Camel 2.9 to use a better syntax parser, which can do index precise error messages, so you know exactly what is wrong and where the problem is. For example if you have made a typo in one of the operators, then previously the parser would not be able to detect this, and cause the evaluation to be true. There are a few changes in the syntax which are no longer backwards compatible. When using Simple language as a Predicate then the literal text must be enclosed in either single or double quotes. For example: "${body} == 'Camel'". Notice how we have single quotes around the literal. The old style of using "body" and "header.foo" to refer to the message body and header is @deprecated, and it is encouraged to always use ${} tokens for the built-in functions.
The range operator now requires the range to be in single quote as well as shown: "${header.zip} between '30000..39999'".

To get the body of the in message: body, or in.body or ${body}.

A complex expression must use ${} placeholders, such as: Hello ${in.header.name} how are you?.

You can have multiple functions in the same expression: "Hello ${in.header.name} this is ${in.header.me} speaking". However you can not nest functions in Camel 2.8.x or older e.g., having another ${} placeholder in an existing, is not allowed. From Camel 2.9 you can nest functions.

Variables

confluenceTableSmall

Variable

Type

Description

camelId

String

Camel 2.10: the CamelContext name.

camelContext.OGNL

Object

Camel 2.11: the CamelContext invoked using a Camel OGNL expression.

collate(group)

List

Camel 2.17: The collate function iterates the message body and groups the data into sub lists of specified size. This can be used with the Splitter EIP to split a message body and group/batch the split sub messages into a group of N sub lists. This method works similar to the collate method in Groovy.

exchange

Exchange

Camel 2.16: the Exchange.

exchange.OGNL

Object

Camel 2.16: the Exchange invoked using a Camel OGNL expression.

exchangeId

String

Camel 2.3: the exchange Id.

id

String

The input message Id.

body

Object

The input body.

in.body

Object

The input body.

body.OGNL

Object

Camel 2.3: the input body invoked using a Camel OGNL expression.

in.body.OGNL

Object

Camel 2.3: the input body invoked using a Camel OGNL expression.

bodyAs(type)

Type

Camel 2.3: Converts the body to the given type determined by its classname. The converted body can be null.

bodyAs(type).OGNL

Object

Camel 2.18: Converts the body to the given type determined by its classname and then invoke methods using a Camel OGNL expression. The converted body can be null.

mandatoryBodyAs(type)

Type

Camel 2.5: Converts the body to the given type determined by its classname, and expects the body to be not null.

mandatoryBodyAs(type).OGNL

Object

Camel 2.18: Converts the body to the given type determined by its classname and then invoke methods using a Camel OGNL expression.

out.body

Object

The output body.

header.foo

Object

Refer to the input foo header.

header[foo]

Object

Camel 2.9.2: refer to the input foo header.

headers.foo

Object

Refer to the input foo header.

headers[foo]

Object

Camel 2.9.2: refer to the input foo header.

in.header.foo

Object

Refer to the input foo header.

in.header[foo]

Object

Camel 2.9.2: refer to the input foo header.

in.headers.foo

Object

Refer to the input foo header.

in.headers[foo]

Object

Camel 2.9.2: refer to the input foo header.

header.foo[bar]

Object

Camel 2.3: regard input foo header as a map and perform lookup on the map with bar as key.

in.header.foo[bar]

Object

Camel 2.3: regard input foo header as a map and perform lookup on the map with bar as key.

in.headers.foo[bar]

Object

Camel 2.3: regard input foo header as a map and perform lookup on the map with bar as key.

header.foo.OGNL

Object

Camel 2.3: refer to the input foo header and invoke its value using a Camel OGNL expression.

in.header.foo.OGNL

Object

Camel 2.3: refer to the input foo header and invoke its value using a Camel OGNL expression.

in.headers.foo.OGNL

Object

Camel 2.3: refer to the input foo header and invoke its value using a Camel OGNL expression.

out.header.foo

Object

Refer to the out header foo.

out.header[foo]

Object

Camel 2.9.2: refer to the out header foo.

out.headers.foo

Object

Refer to the out header foo.

out.headers[foo]

Object

Camel 2.9.2: refer to the out header foo.

headerAs(key,type)

Type

Camel 2.5: Converts the header to the given type determined by its classname.

headers

Map

Camel 2.9: refer to the input headers.

in.headers

Map

Camel 2.9: refer to the input headers.

property.foo

Object

Deprecated: refer to the foo property on the exchange.

exchangeProperty.foo

Object

Camel 2.15: refer to the foo property on the exchange.

property[foo]

Object

Deprecated: refer to the foo property on the exchange.

exchangeProperty[foo]

Object

Camel 2.15: refer to the foo property on the exchange.

property.foo.OGNL

Object

Deprecated: refer to the foo property on the exchange and invoke its value using a Camel OGNL expression.

exchangeProperty.foo.OGNL

Object

Camel 2.15: refer to the foo property on the exchange and invoke its value using a Camel OGNL expression.

sys.foo

String

Refer to the system property foo.

sysenv.foo

String

Camel 2.3: refer to the system environment property foo.

exception

Object

Camel 2.4: Refer to the exception object on the exchange, is null if no exception set on exchange. Will fallback and grab caught exceptions (Exchange.EXCEPTION_CAUGHT) if the Exchange has any.

exception.OGNL

Object

Camel 2.4: Refer to the exchange exception invoked using a Camel OGNL expression object

exception.message

String

Refer to the exception.message on the exchange, is null if no exception set on exchange. Will fallback and grab caught exceptions (Exchange.EXCEPTION_CAUGHT) if the Exchange has any.

exception.stacktrace

String

Camel 2.6. Refer to the exception.stracktrace on the exchange. Result is null if no exception set on exchange. Will fallback and grab caught exceptions (Exchange.EXCEPTION_CAUGHT) if the Exchange has any.

date:command:pattern

String

Date formatting using the java.text.SimpleDateFormat patterns. Supported commands are: 

  • now for current timestamp.

  • in.header.xxx or header.xxx to use the Date object in the IN header with the key xxx.

  • out.header.xxx to use the Date object in the OUT header with the key xxx.

bean:bean expression

Object

Invoking a bean expression using the Bean language. Specifying a method name you must use dot as separator. We also support the ?method=methodname syntax that is used by the Bean component.

properties:locations:key

String

Deprecated: (use properties-location instead) Camel 2.3: Lookup a property with the given key. The locations option is optional. See more at Using PropertyPlaceholder.

properties-location:locations:key

String

Camel 2.14.1: Lookup a property with the given key. The locations option is optional. See more at Using PropertyPlaceholder.

properties:key:default

String

Camel 2.14.1: Lookup a property with the given key. If the key does not exists or has no value, then an optional default value can be specified.

routeId

String

Camel 2.11: Returns the Id of the current route the Exchange is being routed.

threadName

String

Camel 2.3: Returns the name of the current thread. Can be used for logging purpose.

ref:xxx

Object

Camel 2.6: To lookup a bean from the Registry with the given Id.

type:name.field

Object

Camel 2.11: To refer to a type or field by its FQN name. To refer to a field you can append .FIELD_NAME. For example you can refer to the constant field from Exchange as: org.apache.camel.Exchange.FILE_NAME

.

null

null

Camel 2.12.3: represents a null.

random(value)

Integer

Camel 2.16.0: returns a random Integer between 0 (included) and value (excluded)

random(min,max)

Integer

Camel 2.16.0: returns a random Integer between min (included) and max (excluded)

skip(number)

Iterator

Camel 2.19: The skip function iterates the message body and skips the first number of items. This can be used with the Splitter EIP to split a message body and skip the first N number of items.

messageHistory

String

Camel 2.17: The message history of the current exchange how it has been routed. This is similar to the route stack-trace message history the error handler logs in case of an unhandled exception.

messageHistory(false)

String

Camel 2.17: As messageHistory but without the exchange details (only includes the route strack-trace). This can be used if you do not want to log sensitive data from the message itself.

OGNL expression support

Available as of Camel 2.3

Camel's OGNL support is for invoking methods only. You cannot access fields.
From Camel 2.11.1: we added special support for accessing the length field of Java arrays.

The Simple and Bean language now supports a Camel OGNL notation for invoking beans in a chain like fashion. Suppose the Message IN body contains a POJO which has a getAddress() method.

Then you can use Camel OGNL notation to access the address object:

javasimple("${body.address}") simple("${body.address.street}") simple("${body.address.zip}")

Camel understands the shorthand names for accessors, but you can invoke any method or use the real name such as:

javasimple("${body.address}") simple("${body.getAddress.getStreet}") simple("${body.address.getZip}") simple("${body.doSomething}")

You can also use the null safe operator (?.) to avoid a NPE if for example the body does not have an address

javasimple("${body?.address?.street}")

It is also possible to index in Map or List types, so you can do:

javasimple("${body[foo].name}")

To assume the body is Map based and lookup the value with foo as key, and invoke the getName method on that value.

key with spaces

If the key has space, then you must enclose the key with quotes, for example:

javasimple("${body['foo bar'].name}")

You can access the Map or List objects directly using their key name (with or without dots) :

javasimple("${body[foo]}") simple("${body[this.is.foo]}")

Suppose there was no value with the key foo then you can use the null safe operator to avoid a NPE as shown:

javasimple("${body[foo]?.name}")

You can also access List types, for example to get lines from the address you can do:

javasimple("${body.address.lines[0]}") simple("${body.address.lines[1]}") simple("${body.address.lines[2]}")

There is a special last keyword which can be used to get the last value from a list.

javasimple("${body.address.lines[last]}")

And to get the penultimate line use subtraction. In this case use last-1 for this:

javasimple("${body.address.lines[last-1]}")

And the third last is of course:

javasimple("${body.address.lines[last-2]}")

And you can call the size method on the list with

javasimple("${body.address.lines.size}")

From Camel 2.11.1 we added support for the length field for Java arrays as well. Example:

javaString[] lines = new String[]{"foo", "bar", "cat"}; exchange.getIn().setBody(lines); simple("There are ${body.length} lines")

And yes you can combine this with the operator support as shown below:

javasimple("${body.address.zip} > 1000")

Operator Support

The parser is limited to only support a single operator. To enable it the left value must be enclosed in ${}.

The syntax is:

java${leftValue} OP rightValue

Where the rightValue can be a String literal enclosed in ' ', null, a constant value or another expression enclosed in ${}.

Important

There must be spaces around the operator.

Camel will automatically type convert the rightValue type to the leftValue type, so it is possible to for example, convert a string into a numeric so you can use > comparison for numeric values.

The following operators are supported:

Operator

Description

==

Equals.

=~

Camel 2.16: equals ignore case (will ignore case when comparing String values).

>

Greater than.

>=

Greater than or equals.

<

Less than.

<=

Less than or equals.

!=

Not equals.

contains

For testing if contains in a string based value.

not contains

For testing if not contains in a string based value.

regex

For matching against a given regular expression pattern defined as a String value.

not regex

For not matching against a given regular expression pattern defined as a String value.

in

For matching if in a set of values, each element must be separated by comma.

If you want to include an empty value, then it must be defined using double comma, eg ',,bronze,silver,gold', which
is a set of four values with an empty value and then the three medals.

not in

For matching if not in a set of values, each element must be separated by comma.

If you want to include an empty value, then it must be defined using double comma. Example: ',,bronze,silver,gold', which
is a set of four values with an empty value and then the three medals.

is

For matching if the left hand side type is an instanceof the value.

not is

For matching if the left hand side type is not an instanceof the value.

range

For matching if the left hand side is within a range of values defined as numbers: from..to.

From Camel 2.9: the range values must be enclosed in single quotes.

not range

For matching if the left hand side is not within a range of values defined as numbers: from..to.

From Camel 2.9: the range values must be enclosed in single quotes.

starts with

Camel 2.17.1, 2.18: For testing if the left hand side string starts with the right hand string.

ends with

Camel 2.17.1, 2.18: For testing if the left hand side string ends with the right hand string.

And the following unary operators can be used:

Operator

Description

++

Camel 2.9: To increment a number by one. The left hand side must be a function, otherwise parsed as literal.

--

Camel 2.9: To decrement a number by one. The left hand side must be a function, otherwise parsed as literal.

\

Camel 2.9.3 to 2.10.x To escape a value, e.g., \$, to indicate a $ sign. Special: Use \n for new line, \t for tab, and \r for carriage return.

Note: Escaping is not supported using the File Language.

Note: from Camel 2.11, the escape character is no longer supported. It has been replaced with the following three escape sequences.

\n

Camel 2.11: To use newline character.

\t

Camel 2.11: To use tab character.

\r

Camel 2.11: To use carriage return character.

\}

Camel 2.18: To use the } character as text.

And the following logical operators can be used to group expressions:

Operator

Description

and

Deprecated: use && instead. The logical and operator is used to group two expressions.

or

Deprecated: use || instead. The logical or operator is used to group two expressions.

&&

Camel 2.9: The logical and operator is used to group two expressions.

||

Camel 2.9: The logical or operator is used to group two expressions.

Using and,or operators

In Camel 2.4 and older the and or or can only be used once in a simple language expression.

From Camel 2.5: you can use these operators multiple times.

The syntax for AND is:

java${leftValue} OP rightValue and ${leftValue} OP rightValue

And the syntax for OR is:

java${leftValue} OP rightValue or ${leftValue} OP rightValue

Some examples:

java// exact equals match simple("${in.header.foo} == 'foo'")   // ignore case when comparing, so if the header has value FOO this will match simple("${in.header.foo} =~ 'foo'") // here Camel will type convert '100' into the type of in.header.bar and if it is an Integer '100' will also be converter to an Integer simple("${in.header.bar} == '100'") simple("${in.header.bar} == 100") // 100 will be converter to the type of in.header.bar so we can do > comparison simple("${in.header.bar} > 100") Comparing with different types

When you compare with different types such as String and int, then you have to take a bit care. Camel will use the type from the left hand side as first priority. And fallback to the right hand side type if both values couldn't be compared based on that type. This means you can flip the values to enforce a specific type. Suppose the bar value above is a String. Then you can flip the equation:

javasimple("100 < ${in.header.bar}")

which then ensures the int type is used as first priority.

This may change in the future if the Camel team improves the binary comparison operations to prefer numeric types over String based. It's most often the String type which causes problem when comparing with numbers.

java// testing for null simple("${in.header.baz} == null") // testing for not null simple("${in.header.baz} != null")

And a bit more advanced example where the right value is another expression,

javasimple("${in.header.date} == ${date:now:yyyyMMdd}") simple("${in.header.type} == ${bean:orderService?method=getOrderType}")

And an example with contains, testing if the title contains the word Camel:

javasimple("${in.header.title} contains 'Camel'")

And an example with regex, testing if the number header is a four digit value:

javasimple("${in.header.number} regex '\\d{4}'")

And finally an example if the header equals any of the values in the list. Each element must be separated by comma, and no space around. This also works for numbers etc, as Camel will convert each element into the type of the left hand side.

javasimple("${in.header.type} in 'gold,silver'")

And for all the last three we also support the negate test using not:

javasimple("${in.header.type} not in 'gold,silver'")

And you can test if the type is a certain instance, e.g., for instance a String:

javasimple("${in.header.type} is 'java.lang.String'")

We have added a shorthand for all java.lang types so you can write it as:

javasimple("${in.header.type} is 'String'")

Ranges are also supported. The range interval requires numbers and both from and end are inclusive. For instance to test whether a value is between 100 and 199:

javasimple("${in.header.number} range 100..199")

Notice we use .. in the range without spaces. It is based on the same syntax as Groovy.

From Camel 2.9: the range value must be in single quotes:

javasimple("${in.header.number} range '100..199'") Can be used in Spring XML

As the Spring XML does not have all the power as the Java DSL with all its various builder methods, you have to resort to use some other languages for testing with simple operators. Now you can do this with the simple language. In the sample below we want to test if the header is a widget order:

xml<from uri="seda:orders"> <filter> <simple>${in.header.type} == 'widget'</simple> <to uri="bean:orderService?method=handleWidget"/> </filter> </from>

Using andor

If you have two expressions you can combine them with the and or or operator.

Camel 2.9 onwards

Use && or ||

For instance:

javasimple("${in.header.title} contains 'Camel' and ${in.header.type'} == 'gold'")

And of course the or is also supported. The sample would be:

javasimple("${in.header.title} contains 'Camel' or ${in.header.type'} == 'gold'")

Note: currently and or or can only be used once in a simple language expression. This might change in the future. So you cannot do:

javasimple("${in.header.title} contains 'Camel' and ${in.header.type'} == 'gold' and ${in.header.number} range 100..200")

Samples

In the Spring XML sample below we filter based on a header value:

xml<from uri="seda:orders"> <filter> <simple>${in.header.foo}</simple> <to uri="mock:fooOrders"/> </filter> </from>

The Simple language can be used for the predicate test above in the Message Filter pattern, where we test if the in message has a foo header (a header with the key foo exists). If the expression evaluates to true then the message is routed to the mock:fooOrders endpoint, otherwise it is lost in the deep blue sea (wink).

The same example in Java DSL:

javafrom("seda:orders") .filter().simple("${in.header.foo}") .to("seda:fooOrders");

You can also use the simple language for simple text concatenations such as:

javafrom("direct:hello") .transform().simple("Hello ${in.header.user} how are you?") .to("mock:reply");

Notice that we must use ${} placeholders in the expression now to allow Camel to parse it correctly.

And this sample uses the date command to output current date.

javafrom("direct:hello") .transform().simple("The today is ${date:now:yyyyMMdd} and it is a great day.") .to("mock:reply");

And in the sample below we invoke the bean language to invoke a method on a bean to be included in the returned string:

javafrom("direct:order") .transform().simple("OrderId: ${bean:orderIdGenerator}") .to("mock:reply");

Where orderIdGenerator is the id of the bean registered in the Registry. If using Spring then it is the Spring bean id.

If we want to declare which method to invoke on the order id generator bean we must prepend .method name such as below where we invoke the generateId method.

javafrom("direct:order") .transform().simple("OrderId: ${bean:orderIdGenerator.generateId}") .to("mock:reply");

We can use the ?method=methodname option that we are familiar with the Bean component itself:

javafrom("direct:order") .transform().simple("OrderId: ${bean:orderIdGenerator?method=generateId}") .to("mock:reply");

From Camel 2.3: you can also convert the body to a given type, for example to ensure that it is a String you can do:

xml<transform> <simple>Hello ${bodyAs(String)} how are you?</simple> </transform>

There are a few types which have a shorthand notation, so we can use String instead of java.lang.String. These are: byte[], String, IntegerLong. All other types must use their FQN name, e.g. org.w3c.dom.Document.

It is also possible to lookup a value from a header Map in Camel 2.3:

xml<transform> <simple>The gold value is ${header.type[gold]}</simple> </transform>

In the code above we lookup the header with name type and regard it as a java.util.Map and we then lookup with the key gold and return the value. If the header is not convertible to Map an exception is thrown. If the header with name type does not exist null is returned.

From Camel 2.9: you can nest functions, such as shown below:

xml<setHeader headerName="myHeader"> <simple>${properties:${header.someKey}}</simple> </setHeader>

Referring to Constants or Enums

Available from Camel 2.11

Suppose you have an enum for customers:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/Customer.java}And in a Content Based Router we can use the Simple language to refer to this enum, to check the message which enum it matches.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/CBRSimpleTypeTest.java}

Using New Lines or Tabs in XML DSLs

Available from Camel 2.9.3

From Camel 2.9.3: it is easier to specify new lines or tabs in XML DSLs as you can escape the value now

xml<transform> <simple>The following text\nis on a new line</simple> </transform>

Leading and Trailing Whitespace Handling

Available from Camel 2.10.0

From Camel 2.10.0: the trim attribute of the expression can be used to control whether the leading and trailing whitespace characters are removed or preserved. The default of trim=true removes all whitespace characters.

xml<setBody> <simple trim="false">You get some trailing whitespace characters. </simple> </setBody>

Setting the Result Type

Available from Camel 2.8

You can now provide a result type to the Simple expression, which means the result of the evaluation will be converted to the desired type. This is most usable to define types such as boolean's, integer's, etc.

For example to set a header as a boolean type you can do:

.setHeader("cool", simple("true", Boolean.class))

And in XML DSL

xml<setHeader headerName="cool"> <!-- use resultType to indicate that the type should be a java.lang.Boolean --> <simple resultType="java.lang.Boolean">true</simple> </setHeader>

Changing Function Start and End Tokens

Available from Camel 2.9.1

You can configure the function start and end tokens - ${} using the setters changeFunctionStartToken and changeFunctionEndToken on SimpleLanguage, using Java code. From Spring XML you can define a <bean> tag with the new changed tokens in the properties as shown below:

xml<!-- configure Simple to use custom prefix/suffix tokens --> <bean id="simple" class="org.apache.camel.language.simple.SimpleLanguage"> <property name="functionStartToken" value="["/> <property name="functionEndToken" value="]"/> </bean>

In the example above we use [] as the changed tokens. Notice by changing the start/end token you change those in all the Camel applications which share the same camel-core on their classpath. For example in an OSGi server this may affect many applications, where as a Web Application as a WAR file it only affects the Web Application.

Loading Script from External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as: classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location, e.g., to refer to a file on the classpath you can do:

java.setHeader("myHeader").simple("resource:classpath:mysimple.txt")

Setting Spring beans to Exchange properties

Available from Camel 2.6

You can set a spring bean into an exchange property as shown below:

xml<bean id="myBeanId" class="my.package.MyCustomClass"/> <route> <!-- ... --> <setProperty propertyName="monitoring.message"> <simple>ref:myBeanId</simple> </setProperty> <!-- ... --> </route>

Dependencies

The Simple language is part of camel-core.

File Expression Language

File language is now merged with Simple language

From Camel 2.2: the file language is now merged with Simple language which means you can use all the file syntax directly within the simple language.

The File Expression Language is an extension to the Simple language, adding file related capabilities. These capabilities are related to common use cases working with file path and names. The goal is to allow expressions to be used with the File and FTP components for setting dynamic file patterns for both consumer and producer.

Syntax

This language is an extension to the Simple language so the Simple syntax applies also. So the table below only lists the additional. By contrast to the Simple language, the File Language also supports the use of Constant expressions to enter a fixed filename, for example.

All the file tokens use the same expression name as the method on the java.io.File object. For example: file:absolute refers to the java.io.File.getAbsolute() method.

Note: not all expressions are supported by the current Exchange. For example, the FTP component supports some of the options, whereas the File component supports all of them.

Expression

Type

File Consumer

File Producer

FTP Consumer

FTP Producer

Description

date:command:pattern

String

yes

yes

yes

yes

For date formatting using the java.text.SimpleDateFormat patterns which is an extension to the Simple language.

Additional command is: file (consumers only) for the last modified timestamp of the file.

Note: all the commands from the Simple language can also be used.

file:absolute

Boolean

yes

no

no

no

Refers to whether the file is regarded as absolute or relative.

file:absolute.path

String

yes

no

no

no

Refers to the absolute file path.

file:ext

String

yes

no

yes

no

Refers to the file extension only.

file:length

Long

yes

no

yes

no

Refers to the file length returned as a Long type.

file:modified

Date

yes

no

yes

no

Refers to the file last modified returned as a Date type.

file:name

String

yes

no

yes

no

Refers to the file name (is relative to the starting directory, see note below).

file:name.ext

String

yes

no

yes

no

Camel 2.3: refers to the file extension only.

file:name.ext.singleStringyesnoyesnoCamel 2.14.4/2.15.3: refers to the file extension. If the file extension has multiple dots, then this expression strips and only returns the last part.
file:name.noext

String

yes

no

yes

no

Refers to the file name with no extension (is relative to the starting directory, see note below).

file:name.noext.singleStringyesnoyesnoCamel 2.14.4/2.15.3: refers to the file name with no extension (is relative to the starting directory, see note below). If the file extension has multiple dots, then this expression strips only the last part, and keep the others.
file:onlyname

String

yes

no

yes

no

Refers to the file name only with no leading paths.

file:onlyname.noext

String

yes

no

yes

no

Refers to the file name only with no extension and with no leading paths.

file:onlyname.noext.singleStringyesnoyesnoCamel 2.14.4/2.15.3: refers to the file name only with no extension and with no leading paths. If the file extension has multiple dots, then this expression strips only the last part, and keep the others.
file:parent

String

yes

no

yes

no

Refers to the file parent.

file:path

String

yes

no

yes

no

Refers to the file path.

file:size

Long

yes

no

yes

no

Camel 2.5: refers to the file length returned as a Long type.

File Token Example

Relative Paths

We have a java.io.File handle for the file hello.txt in the following relative directory: .\filelanguage\test. And we configure our endpoint to use this starting directory .\filelanguage.

The file tokens returned are:

Expression

Returns

file:absolute

false

file:absolute.path

\workspace\camel\camel-core\target\filelanguage\test\hello.txt

file:ext

txt

file:name

test\hello.txt

file:name.ext

txt

file:name.noext

test\hello

file:onlyname

hello.txt

file:onlyname.noext

hello

file:parent

filelanguage\test

file:path

filelanguage\test\hello.txt

Absolute Paths

We have a java.io.File handle for the file hello.txt in the following absolute directory: \workspace\camel\camel-core\target\filelanguage\test. And we configure out endpoint to use the absolute starting directory: \workspace\camel\camel-core\target\filelanguage.

The file tokens return are:

Expression

Returns

file:absolute

true

file:absolute.path

\workspace\camel\camel-core\target\filelanguage\test\hello.txt

file:ext

txt

file:name

test\hello.txt

file:name.ext

txt

file:name.noext

test\hello

file:onlyname

hello.txt

file:onlyname.noext

hello

file:parent

\workspace\camel\camel-core\target\filelanguage\test

file:path

\workspace\camel\camel-core\target\filelanguage\test\hello.txt

Examples

You can enter a fixed Constant expression such as myfile.txt:

fileName="myfile.txt"

Lets assume we use the file consumer to read files and want to move the read files to backup folder with the current date as a sub folder. This can be achieved using an expression like:

fileName="backup/${date:now:yyyyMMdd}/${file:name.noext}.bak"

relative folder names are also supported so suppose the backup folder should be a sibling folder then you can append .. as:

fileName="../backup/${date:now:yyyyMMdd}/${file:name.noext}.bak"

As this is an extension to the Simple language we have access to all the goodies from this language also, so in this use case we want to use the in.header.type as a parameter in the dynamic expression:

fileName="../backup/${date:now:yyyyMMdd}/type-${in.header.type}/backup-of-${file:name.noext}.bak"

If you have a custom Date you want to use in the expression then Camel supports retrieving dates from the message header.

fileName="orders/order-${in.header.customerId}-${date:in.header.orderDate:yyyyMMdd}.xml"

And finally we can also use a bean expression to invoke a POJO class that generates some String output (or convertible to String) to be used:

fileName="uniquefile-${bean:myguidgenerator.generateid}.txt"

And of course all this can be combined in one expression where you can use the File Language, Simple and the Bean language in one combined expression. This is pretty powerful for those common file path patterns.

Using Spring's PropertyPlaceholderConfigurer with the File Component

In Camel you can use the File Language directly from the Simple language which makes a Content Based Router easier to do in Spring XML, where we can route based on file extensions as shown below:

<from uri="file://input/orders"/>
  <choice>
    <when>
      <simple>${file:ext} == 'txt'</simple>
      <to uri="bean:orderService?method=handleTextFiles"/>
    </when>
    <when>
      <simple>${file:ext} == 'xml'</simple>
      <to uri="bean:orderService?method=handleXmlFiles"/>
    </when>
    <otherwise>
      <to uri="bean:orderService?method=handleOtherFiles"/>
    </otherwise>
  </choice>

If you use the fileName option on the File endpoint to set a dynamic filename using the File Language then make sure you use the alternative syntax (available from Camel 2.5) to avoid clashing with Spring's PropertyPlaceholderConfigurer.

bundle-context.xml
<bean id="propertyPlaceholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
  <property name="location" value="classpath:bundle-context.cfg"/>
</bean>

<bean id="sampleRoute" class="SampleRoute">
  <property name="fromEndpoint" value="${fromEndpoint}"/>
  <property name="toEndpoint" value="${toEndpoint}"/>
</bean>
bundle-context.cfg
fromEndpoint=activemq:queue:test
toEndpoint=file://fileRoute/out?fileName=test-$simple{date:now:yyyyMMdd}.txt

Notice how we use the $simple{} syntax in the toEndpoint above. If you don't do this, they will clash and Spring will throw an exception:

org.springframework.beans.factory.BeanDefinitionStoreException:
Invalid bean definition with name 'sampleRoute' defined in class path resource [bundle-context.xml]:
Could not resolve placeholder 'date:now:yyyyMMdd'

Dependencies

The File language is part of camel-core.

SQL Language

The SQL support is added by JoSQL and is primarily used for performing SQL queries on in-memory objects. If you prefer to perform actual database queries then check out the JPA component.

Looking for the SQL component

Camel has both a SQL language and a SQL Component. This page is about the SQL language. Click on SQL Component if you are looking for the component instead.

To use SQL in your camel routes you need to add the a dependency on camel-josql which implements the SQL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-josql</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

Camel supports SQL to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use SQL to create an Predicate in a Message Filter or as an Expression for a Recipient List.

from("queue:foo").setBody().sql("select * from MyType").to("queue:bar")

And the spring DSL:

   <from uri="queue:foo"/>
   <setBody>
       <sql>select * from MyType</sql>
   </setBody>
   <to uri="queue:bar"/>

Variables

Variable

Type

Description

exchange

Exchange

the Exchange object

in

Message

the exchange.in message

out

Message

the exchange.out message

the property key

Object

the Exchange properties

the header key

Object

the exchange.in headers

the variable key

Object

if any additional variables is added using setVariables method

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

.setHeader("myHeader").sql("resource:classpath:mysql.sql")

XPath

Camel supports XPath to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use XPath to create an Predicate in a Message Filter or as an Expression for a Recipient List.

Streams

If the message body is stream based, which means the input is received by Camel as a stream, then you will only be able to read the content of the stream once. Oftentimes when using XPath as Message Filter or Content Based Router the data will be accessed multiple times. Therefore use Stream caching or convert the message body to a String beforehand. This makes it safe to be re-read multiple times.

from("queue:foo") .filter().xpath("//foo")) .to("queue:bar") from("queue:foo") .choice().xpath("//foo")).to("queue:bar") .otherwise().to("queue:others");

Namespaces

You can easily use namespaces with XPath expressions using the Namespaces helper class.{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/XPathWithNamespacesFilterTest.java}

Variables

Variables in XPath is defined in different namespaces. The default namespace is http://camel.apache.org/schema/spring.

Namespace URI

Local part

Type

Description

http://camel.apache.org/xml/in/

in

Message

The exchange.in message.

http://camel.apache.org/xml/out/

out

Message

The exchange.out message.

http://camel.apache.org/xml/function/

functions

Object

Camel 2.5: Additional functions.

http://camel.apache.org/xml/variables/environment-variables

env

Object

OS environment variables.

http://camel.apache.org/xml/variables/system-properties

system

Object

Java System properties.

http://camel.apache.org/xml/variables/exchange-property

 

Object

The exchange property.

Camel will resolve variables according to either:

  • namespace given
  • no namespace given

Namespace Given

If the namespace is given then Camel is instructed exactly what to return. However when resolving either IN or OUT Camel will try to resolve a header with the given local part first, and return it. If the local part has the value body then the body is returned instead.

No Namespace Given

If there is no namespace given then Camel resolves only based on the local part. Camel will try to resolve a variable in the following steps:

  • From variables that has been set using the variable(name, value) fluent builder.
  • From message.in.header if there is a header with the given key.
  • From exchange.properties if there is a property with the given key.

Functions

Camel adds the following XPath functions that can be used to access the exchange:

Function

Argument

Type

Description

in:body

none

Object

Will return the IN message body.

in:header

the header name

Object

Will return the IN message header.

out:body

none

Object

Will return the OUT message body.

out:header

the header name

Object

Will return the OUT message header.

function:properties

key for property

String

Camel 2.5: To lookup a property using the Properties component (property placeholders).

function:simple

simple expression

Object

Camel 2.5: To evaluate a Simple expression.

Note: function:properties and function:simple is not supported when the return type is a NodeSet, such as when using with a Splitter EIP.

Here's an example showing some of these functions in use.{snippet:id=ex|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/language/XPathFunctionTest.java}And the new functions introduced in Camel 2.5:{snippet:id=ex|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/xml/XPathFunctionsTest.java}

Using XML Configuration

If you prefer to configure your routes in your Spring XML file then you can use XPath expressions as follows

xml<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring" xmlns:foo="http://example.com/person"> <route> <from uri="activemq:MyQueue"/> <filter> <xpath>/foo:person[@name='James']</xpath> <to uri="mqseries:SomeOtherQueue"/> </filter> </route> </camelContext> </beans>

Notice how we can reuse the namespace prefixes, foo in this case, in the XPath expression for easier namespace based XPath expressions! See also this discussion on the mailinglist about using your own namespaces with XPath.

Setting the Result Type

The XPath expression will return a result type using native XML objects such as org.w3c.dom.NodeList. But many times you want a result type to be a String. To do this you have to instruct the XPath which result type to use.

In Java DSL:

javaxpath("/foo:person/@id", String.class)

In Spring DSL you use the resultType attribute to provide a fully qualified classname:

xml<xpath resultType="java.lang.String">/foo:person/@id</xpath>

In @XPath:
Available as of Camel 2.1

java@XPath(value = "concat('foo-',//order/name/)", resultType = String.class) String name)

Where we use the XPath function concat to prefix the order name with foo-. In this case we have to specify that we want a String as result type so the concat function works.

Using XPath on Headers

Available as of Camel 2.11

Some users may have XML stored in a header. To apply an XPath statement to a header's value you can do this by defining the headerName attribute.

In XML DSL:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-test-blueprint/src/test/resources/org/apache/camel/test/blueprint/xpath/XPathHeaderNameTest.xml}And in Java DSL you specify the headerName as the 2nd parameter as shown:

javaxpath("/invoice/@orderType = 'premium'", "invoiceDetails")

Examples

Here is a simple example using an XPath expression as a predicate in a Message Filter{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/XPathFilterTest.java}If you have a standard set of namespaces you wish to work with and wish to share them across many different XPath expressions you can use the NamespaceBuilder as shown in this example{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/XPathWithNamespaceBuilderFilterTest.java}In this sample we have a choice construct. The first choice evaulates if the message has a header key type that has the value Camel. The 2nd choice evaluates if the message body has a name tag <name> which values is Kong.
If neither is true the message is routed in the otherwise block:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/xml/XPathHeaderTest.java}And the spring XML equivalent of the route:{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringXPathHeaderTest-context.xml}

XPath Injection

You can use Bean Integration to invoke a method on a bean and use various languages such as XPath to extract a value from the message and bind it to a method parameter.

The default XPath annotation has SOAP and XML namespaces available. If you want to use your own namespace URIs in an XPath expression you can use your own copy of the XPath annotation to create whatever namespace prefixes you want to use.{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/component/xslt/MyXPath.java}e.g., cut and paste upper code to your own project in a different package and/or annotation name then add whatever namespace prefix/URIs you want in scope when you use your annotation on a method parameter. Then when you use your annotation on a method parameter all the namespaces you want will be available for use in your XPath expression.

Example:

javapublic class Foo { @MessageDriven(uri = "activemq:my.queue") public void doSomething(@MyXPath("/ns1:foo/ns2:bar/text()") String correlationID, @Body String body) { // process the inbound message here } }

Using XPathBuilder Without an Exchange

Available as of Camel 2.3

You can now use the org.apache.camel.builder.XPathBuilder without the need for an Exchange. This comes handy if you want to use it as a helper to do custom XPath evaluations. It requires that you pass in a CamelContext since a lot of the moving parts inside the XPathBuilder requires access to the Camel Type Converter and hence why CamelContext is needed.

For example you can do something like this:

javaboolean matches = XPathBuilder.xpath("/foo/bar/@xyz").matches(context, "<foo><bar xyz='cheese'/></foo>"));

This will match the given predicate.

You can also evaluate for example as shown in the following three examples:

javaString name = XPathBuilder.xpath("foo/bar").evaluate(context, "<foo><bar>cheese</bar></foo>", String.class); Integer number = XPathBuilder.xpath("foo/bar").evaluate(context, "<foo><bar>123</bar></foo>", Integer.class); Boolean bool = XPathBuilder.xpath("foo/bar").evaluate(context, "<foo><bar>true</bar></foo>", Boolean.class);

Evaluating with a String result is a common requirement and thus you can do it a bit simpler:

String name = XPathBuilder.xpath("foo/bar").evaluate(context, "<foo><bar>cheese</bar></foo>");

Using Saxon with XPathBuilder

Available as of Camel 2.3

You need to add camel-saxon as dependency to your project. It's now easier to use Saxon with the XPathBuilder which can be done in several ways as shown below. Where as the latter ones are the easiest ones.

Using a factory{snippet:id=e1|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XPathTest.java}Using the object model
{snippet:id=e2|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XPathTest.java}The easy one{snippet:id=e3|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XPathTest.java}

Setting a Custom XPathFactory Using System Property

Available as of Camel 2.3

Camel now supports reading the JVM system property javax.xml.xpath.XPathFactory that can be used to set a custom XPathFactory to use.

This unit test shows how this can be done to use Saxon instead:{snippet:id=e4|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XPathTest.java}Camel will log at INFO level if it uses a non default XPathFactory such as:

XPathBuilder INFO Using system property javax.xml.xpath.XPathFactory:http://saxon.sf.net/jaxp/xpath/om with value: net.sf.saxon.xpath.XPathFactoryImpl when creating XPathFactory

To use Apache Xerces you can configure the system property:

-Djavax.xml.xpath.XPathFactory=org.apache.xpath.jaxp.XPathFactoryImpl

Enabling Saxon from Spring DSL

Available as of Camel 2.10

Similarly to Java DSL, to enable Saxon from Spring DSL you have three options:

Specifying the factory

xml<xpath factoryRef="saxonFactory" resultType="java.lang.String">current-dateTime()</xpath>

Specifying the object model

xml<xpath objectModel="http://saxon.sf.net/jaxp/xpath/om" resultType="java.lang.String">current-dateTime()</xpath>

Shortcut

xml<xpath saxon="true" resultType="java.lang.String">current-dateTime()</xpath>

Namespace Auditing to Aid Debugging

Available as of Camel 2.10

A large number of XPath-related issues that users frequently face are linked to the usage of namespaces. You may have some misalignment between the namespaces present in your message and those that your XPath expression is aware of or referencing. XPath predicates or expressions that are unable to locate the XML elements and attributes due to namespaces issues may simply look like "they are not working", when in reality all there is to it is a lack of namespace definition.

Namespaces in XML are completely necessary, and while we would love to simplify their usage by implementing some magic or voodoo to wire namespaces automatically, truth is that any action down this path would disagree with the standards and would greatly hinder interoperability.

Therefore, the utmost we can do is assist you in debugging such issues by adding two new features to the XPath Expression Language and are thus accessible from both predicates and expressions.

Logging the Namespace Context of Your XPath Expression/Predicate

Every time a new XPath expression is created in the internal pool, Camel will log the namespace context of the expression under the org.apache.camel.builder.xml.XPathBuilder logger. Since Camel represents Namespace Contexts in a hierarchical fashion (parent-child relationships), the entire tree is output in a recursive manner with the following format:

[me: {prefix -> namespace}, {prefix -> namespace}], [parent: [me: {prefix -> namespace}, {prefix -> namespace}], [parent: [me: {prefix -> namespace}]]]

Any of these options can be used to activate this logging:

  1. Enable TRACE logging on the org.apache.camel.builder.xml.XPathBuilder logger, or some parent logger such as org.apache.camel or the root logger.
  2. Enable the logNamespaces option as indicated in Auditing Namespaces, in which case the logging will occur on the INFO level.

AuditingNamespaces

Auditing namespaces

Camel is able to discover and dump all namespaces present on every incoming message before evaluating an XPath expression, providing all the richness of information you need to help you analyse and pinpoint possible namespace issues. To achieve this, it in turn internally uses another specially tailored XPath expression to extract all namespace mappings that appear in the message, displaying the prefix and the full namespace URI(s) for each individual mapping.

Some points to take into account:

  • The implicit XML namespace (xmlns:xml="http://www.w3.org/XML/1998/namespace") is suppressed from the output because it adds no value.
  • Default namespaces are listed under the DEFAULT keyword in the output.
  • Keep in mind that namespaces can be remapped under different scopes. Think of a top-level 'a' prefix which in inner elements can be assigned a different namespace, or the default namespace changing in inner scopes. For each discovered prefix, all associated URIs are listed.

You can enable this option in Java DSL and Spring DSL.

Java DSL:

javaXPathBuilder.xpath("/foo:person/@id", String.class).logNamespaces()

Spring DSL:

xml<xpath logNamespaces="true" resultType="String">/foo:person/@id</xpath>

The result of the auditing will be appear at the INFO level under the org.apache.camel.builder.xml.XPathBuilder logger and will look like the following:

2012-01-16 13:23:45,878 [stSaxonWithFlag] INFO XPathBuilder - Namespaces discovered in message: {xmlns:a=[http://apache.org/camel], DEFAULT=[http://apache.org/default], xmlns:b=[http://apache.org/camelA, http://apache.org/camelB]}

Loading Script from External Resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as: classpath:, file: or http:.
This is done using the following syntax: resource:scheme:location, e.g., to refer to a file on the classpath you can do:

.setHeader("myHeader").xpath("resource:classpath:myxpath.txt", String.class)

Dependencies

The XPath language is part of camel-core.

XQuery

Camel supports XQuery to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use XQuery to create an Predicate in a Message Filter or as an Expression for a Recipient List.

Options

confluenceTableSmall

Name

Default Value

Description

allowStAX

false

Camel 2.8.3/2.9: Whether to allow using StAX as the javax.xml.transform.Source.

Examples

from("queue:foo").filter(). xquery("//foo"). to("queue:bar")

You can also use functions inside your query, in which case you need an explicit type conversion (or you will get a org.w3c.dom.DOMException: HIERARCHY_REQUEST_ERR) by passing the Class as a second argument to the xquery() method.

from("direct:start"). recipientList().xquery("concat('mock:foo.', /person/@city)", String.class);

Variables

The IN message body will be set as the contextItem. Besides this these Variables is also added as parameters:

Variable

Type

Description

exchange

Exchange

The current Exchange

in.body

Object

The In message's body

out.body

Object

The OUT message's body (if any)

in.headers.*

Object

You can access the value of exchange.in.headers with key foo by using the variable which name is in.headers.foo

out.headers.*

Object

You can access the value of exchange.out.headers with key foo by using the variable which name is out.headers.foo variable

key name

Object

Any exchange.properties and exchange.in.headers and any additional parameters set using setParameters(Map). These parameters is added with they own key name, for instance if there is an IN header with the key name foo then its added as foo.

Using XML configuration

If you prefer to configure your routes in your Spring XML file then you can use XPath expressions as follows

xml<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:foo="http://example.com/person" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring"> <route> <from uri="activemq:MyQueue"/> <filter> <xquery>/foo:person[@name='James']</xquery> <to uri="mqseries:SomeOtherQueue"/> </filter> </route> </camelContext> </beans>

Notice how we can reuse the namespace prefixes, foo in this case, in the XPath expression for easier namespace based XQuery expressions!

When you use functions in your XQuery expression you need an explicit type conversion which is done in the xml configuration via the @type attribute:

xml <xquery type="java.lang.String">concat('mock:foo.', /person/@city)</xquery>

Using XQuery as transformation

We can do a message translation using transform or setBody in the route, as shown below:

from("direct:start"). transform().xquery("/people/person");

Notice that xquery will use DOMResult by default, so if we want to grab the value of the person node, using text() we need to tell xquery to use String as result type, as shown:

from("direct:start"). transform().xquery("/people/person/text()", String.class);

 

Using XQuery as an endpoint

Sometimes an XQuery expression can be quite large; it can essentally be used for Templating. So you may want to use an XQuery Endpoint so you can route using XQuery templates.

The following example shows how to take a message of an ActiveMQ queue (MyQueue) and transform it using XQuery and send it to MQSeries.

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="activemq:MyQueue"/> <to uri="xquery:com/acme/someTransform.xquery"/> <to uri="mqseries:SomeOtherQueue"/> </route> </camelContext>

Examples

Here is a simple example using an XQuery expression as a predicate in a Message Filter

{snippet:id=example|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XQueryFilterTest.java}

This example uses XQuery with namespaces as a predicate in a Message Filter

{snippet:id=example|lang=java|url=camel/trunk/components/camel-saxon/src/test/java/org/apache/camel/builder/saxon/XQueryWithNamespacesFilterTest.java}

Learning XQuery

XQuery is a very powerful language for querying, searching, sorting and returning XML. For help learning XQuery try these tutorials

You might also find the XQuery function reference useful

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

.setHeader("myHeader").xquery("resource:classpath:myxquery.txt", String.class)

Dependencies

To use XQuery in your camel routes you need to add the a dependency on camel-saxon which implements the XQuery language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-saxon</artifactId> <version>x.x.x</version> </dependency>

Data Format Appendix

Data Format

Camel supports a pluggable DataFormat to allow messages to be marshalled to and from binary or text formats to support a kind of Message Translator.

The following data formats are currently supported:

  • Object/XML/Webservice marshalling
  • Direct JSON / XML marshalling

And related is the following:

Unmarshalling

If you receive a message from one of the Camel Components such as File, HTTP or JMS you often want to unmarshal the payload into some bean so that you can process it using some Bean Integration or perform Predicate evaluation and so forth. To do this use the unmarshal word in the DSL in Java or the Xml Configuration.

For example

DataFormat jaxb = new JaxbDataFormat("com.acme.model");

from("activemq:My.Queue").
  unmarshal(jaxb).
  to("mqseries:Another.Queue");

The above uses a named DataFormat of jaxb which is configured with a number of Java package names. You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file.

You can also use the DSL itself to define the data format as you use it. For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to ActiveMQ

from("file://foo/bar").
  unmarshal().serialization().
  to("activemq:Some.Queue");

Marshalling

Marshalling is the opposite of unmarshalling, where a bean is marshalled into some binary or textual format for transmission over some transport via a Camel Component. Marshalling is used in the same way as unmarshalling above; in the DSL you can use a DataFormat instance, you can configure the DataFormat dynamically using the DSL or you can refer to a named instance of the format in the Registry.

The following example unmarshals via serialization then marshals using a named JAXB data format to perform a kind of Message Translator

from("file://foo/bar").
  unmarshal().serialization(). 
  marshal("jaxb").
  to("activemq:Some.Queue");

Using Spring XML

This example shows how to configure the data type just once and reuse it on multiple routes

Error formatting macro: snippet: java.lang.NullPointerException

You can also define reusable data formats as Spring beans

<bean id="myJaxb" class="org.apache.camel.model.dataformat.JaxbDataFormat">
  <property name="prettyPrint" value="true"/>
  <property name="contextPath" value="org.apache.camel.example"/>
</bean>  

Serialization

Serialization is a Data Format which uses the standard Java Serialization mechanism to unmarshal a binary payload into Java objects or to marshal Java objects into a binary blob.
For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to ActiveMQ

from("file://foo/bar").
  unmarshal().serialization().
  to("activemq:Some.Queue");

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

JAXB

JAXB is a Data Format which uses the JAXB2 XML marshalling standard which is included in Java 6 to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

Using the Java DSL

For example the following uses a named DataFormat of jaxb which is configured with a number of Java package names to initialize the JAXBContext.

DataFormat jaxb = new JaxbDataFormat("com.acme.model"); from("activemq:My.Queue"). unmarshal(jaxb). to("mqseries:Another.Queue");

You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g.

from("activemq:My.Queue"). unmarshal("myJaxbDataType"). to("mqseries:Another.Queue");

Using Spring XML

The following example shows how to use JAXB to unmarshal using Spring configuring the jaxb data type{snippet:id=example|lang=xml|url=camel/trunk/components/camel-jaxb/src/test/resources/org/apache/camel/example/springDataFormat.xml}This example shows how to configure the data type just once and reuse it on multiple routes.{snippet:id=example|lang=xml|url=camel/trunk/components/camel-jaxb/src/test/resources/org/apache/camel/example/marshalAndUnmarshalWithRef.xml}

Multiple context paths

It is possible to use this data format with more than one context path. You can specify context path using : as separator, for example com.mycompany:com.mycompany2. Note that this is handled by JAXB implementation and might change if you use different vendor than RI.

Partial marshalling/unmarshalling

This feature is new to Camel 2.2.0.
JAXB 2 supports marshalling and unmarshalling XML tree fragments. By default JAXB looks for @XmlRootElement annotation on given class to operate on whole XML tree. This is useful but not always - sometimes generated code does not have @XmlRootElement annotation, sometimes you need unmarshall only part of tree.
In that case you can use partial unmarshalling. To enable this behaviours you need set property partClass. Camel will pass this class to JAXB's unmarshaler.{snippet:id=example|lang=xml|url=camel/trunk/components/camel-jaxb/src/test/resources/org/apache/camel/example/springDataFormatPartial.xml}For marshalling you have to add partNamespace attribute with QName of destination namespace. Example of Spring DSL you can find above.

Fragment

This feature is new to Camel 2.8.0.
JaxbDataFormat has new property fragment which can set the the Marshaller.JAXB_FRAGMENT encoding property on the JAXB Marshaller. If you don't want the JAXB Marshaller to generate the XML declaration, you can set this option to be true. The default value of this property is false.

Ignoring the NonXML Character

This feature is new to Camel 2.2.0.
JaxbDataFromat supports to ignore the NonXML Character, you just need to set the filterNonXmlChars property to be true, JaxbDataFormat will replace the NonXML character with " " when it is marshaling or unmarshaling the message. You can also do it by setting the Exchange property Exchange.FILTER_NON_XML_CHARS.

 

JDK 1.5

JDK 1.6+

Filtering in use

StAX API and implementation

No

Filtering not in use

StAX API only

No

This feature has been tested with Woodstox 3.2.9 and Sun JDK 1.6 StAX implementation.

New for Camel 2.12.1
JaxbDataFormat now allows you to customize the XMLStreamWriter used to marshal the stream to XML. Using this configuration, you can add your own stream writer to completely remove, escape, or replace non-xml characters.

java JaxbDataFormat customWriterFormat = new JaxbDataFormat("org.apache.camel.foo.bar"); customWriterFormat.setXmlStreamWriterWrapper(new TestXmlStreamWriter());

The following example shows using the Spring DSL and also enabling Camel's NonXML filtering:

xml<bean id="testXmlStreamWriterWrapper" class="org.apache.camel.jaxb.TestXmlStreamWriter"/> <jaxb filterNonXmlChars="true" contextPath="org.apache.camel.foo.bar" xmlStreamWriterWrapper="#testXmlStreamWriterWrapper" />

Working with the ObjectFactory

If you use XJC to create the java class from the schema, you will get an ObjectFactory for you JAXB context. Since the ObjectFactory uses JAXBElement to hold the reference of the schema and element instance value, jaxbDataformat will ignore the JAXBElement by default and you will get the element instance value instead of the JAXBElement object form the unmarshaled message body.
If you want to get the JAXBElement object form the unmarshaled message body, you need to set the JaxbDataFormat object's ignoreJAXBElement property to be false.

Setting encoding

You can set the encoding option to use when marshalling. Its the Marshaller.JAXB_ENCODING encoding property on the JAXB Marshaller.
You can setup which encoding to use when you declare the JAXB data format. You can also provide the encoding in the Exchange property Exchange.CHARSET_NAME. This property will overrule the encoding set on the JAXB data format.

In this Spring DSL we have defined to use iso-8859-1 as the encoding:{snippet:id=example|lang=xml|url=camel/trunk/components/camel-jaxb/src/test/resources/org/apache/camel/example/springDataFormatWithEncoding.xml}

Controlling namespace prefix mapping

Available as of Camel 2.11

When marshalling using JAXB or SOAP then the JAXB implementation will automatic assign namespace prefixes, such as ns2, ns3, ns4 etc. To control this mapping, Camel allows you to refer to a map which contains the desired mapping.

Notice this requires having JAXB-RI 2.1 or better (from SUN) on the classpath, as the mapping functionality is dependent on the implementation of JAXB, whether its supported.

For example in Spring XML we can define a Map with the mapping. In the mapping file below, we map SOAP to use soap as prefix. While our custom namespace "http://www.mycompany.com/foo/2" is not using any prefix.

xml <util:map id="myMap"> <entry key="http://www.w3.org/2003/05/soap-envelope" value="soap"/> <!-- we dont want any prefix for our namespace --> <entry key="http://www.mycompany.com/foo/2" value=""/> </util:map>

To use this in JAXB or SOAP you refer to this map, using the namespacePrefixRef attribute as shown below. Then Camel will lookup in the Registry a java.util.Map with the id "myMap", which was what we defined above.

xml <marshal> <soapjaxb version="1.2" contextPath="com.mycompany.foo" namespacePrefixRef="myMap"/> </marshal>

Schema validation

Available as of Camel 2.11

The JAXB Data Format supports validation by marshalling and unmarshalling from/to XML. Your can use the prefix classpath:, file:* or *http: to specify how the resource should by resolved. You can separate multiple schema files by using the ',' character.

Known issue

Camel 2.11.0 and 2.11.1 has a known issue by validation multiple Exchange's in parallel. See CAMEL-6630. This is fixed with Camel 2.11.2/2.12.0.

Using the Java DSL, you can configure it in the following way:

javaJaxbDataFormat jaxbDataFormat = new JaxbDataFormat(); jaxbDataFormat.setContextPath(Person.class.getPackage().getName()); jaxbDataFormat.setSchema("classpath:person.xsd,classpath:address.xsd");

You can do the same using the XML DSL:

xml<marshal> <jaxb id="jaxb" schema="classpath:person.xsd,classpath:address.xsd"/> </marshal>

Camel will create and pool the underling SchemaFactory instances on the fly, because the SchemaFactory shipped with the JDK is not thread safe.
However, if you have a SchemaFactory implementation which is thread safe, you can configure the JAXB data format to use this one:

javaJaxbDataFormat jaxbDataFormat = new JaxbDataFormat(); jaxbDataFormat.setSchemaFactory(thradSafeSchemaFactory);

Schema Location

Available as of Camel 2.14

The JAXB Data Format supports to specify the SchemaLocation when marshaling the XML. 

Using the Java DSL, you can configure it in the following way:

javaJaxbDataFormat jaxbDataFormat = new JaxbDataFormat(); jaxbDataFormat.setContextPath(Person.class.getPackage().getName()); jaxbDataFormat.setSchemaLocation("schema/person.xsd");

You can do the same using the XML DSL:

xml<marshal> <jaxb id="jaxb" schemaLocation="schema/person.xsd"/> </marshal>

Marshal data that is already XML

Available as of Camel 2.14.1

The JAXB marshaller requires that the message body is JAXB compatible, eg its a JAXBElement, eg a java instance that has JAXB annotations, or extend JAXBElement. There can be situations where the message body is already in XML, eg from a String type. There is a new option mustBeJAXBElement you can set to false, to relax this check, so the JAXB marshaller only attempts to marshal JAXBElements (javax.xml.bind.JAXBIntrospector#isElement returns true). And in those situations the marshaller fallbacks to marshal the message body as-is.

XmlRootElement objects

Available as of Camel 2.17.2

The JAXB Data Format option objectFactory has a default value equals to false. This is related to a performance degrading. For more information look at the issue CAMEL-10043

For the marshalling of non-XmlRootElement JaxB objects you'll need to call JaxbDataFormat#setObjectFactory(true)

Dependencies

To use JAXB in your camel routes you need to add the a dependency on camel-jaxb which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jaxb</artifactId> <version>x.x.x</version> </dependency>

XmlBeans

XmlBeans is a Data Format which uses the XmlBeans library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

from("activemq:My.Queue").
  unmarshal().xmlBeans().
  to("mqseries:Another.Queue");

Dependencies

To use XmlBeans in your camel routes you need to add the dependency on camel-xmlbeans which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-xmlbeans</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

XStream

XStream is a Data Format which uses the XStream library to marshal and unmarshal Java objects to and from XML.

To use XStream in your camel routes you need to add the a dependency on camel-xstream which implements this data format.

Maven users will need to add the following dependency to their pom.xml for this component:

xml<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>x.x.x</version> <!-- use the same version as your Camel core version --> </dependency>

Using the Java DSL

// lets turn Object messages into XML then send to MQSeries from("activemq:My.Queue"). marshal().xstream(). to("mqseries:Another.Queue");

If you would like to configure the XStream instance used by the Camel for the message transformation, you can simply pass a reference to that instance on the DSL level.

XStream xStream = new XStream(); xStream.aliasField("money", PurchaseOrder.class, "cash"); // new Added setModel option since Camel 2.14 xStream.setModel("NO_REFERENCES"); ... from("direct:marshal"). marshal(new XStreamDataFormat(xStream)). to("mock:marshaled");

XMLInputFactory and XMLOutputFactory

The XStream library uses the javax.xml.stream.XMLInputFactory and javax.xml.stream.XMLOutputFactory, you can control which implementation of this factory should be used.

The Factory is discovered using this algorithm:
1. Use the javax.xml.stream.XMLInputFactory , javax.xml.stream.XMLOutputFactory system property.
2. Use the lib/xml.stream.properties file in the JRE_HOME directory.
3. Use the Services API, if available, to determine the classname by looking in the META-INF/services/javax.xml.stream.XMLInputFactory, META-INF/services/javax.xml.stream.XMLOutputFactory files in jars available to the JRE.
4. Use the platform default XMLInputFactory,XMLOutputFactory instance.

How to set the XML encoding in Xstream DataFormat?

From Camel 2.2.0, you can set the encoding of XML in Xstream DataFormat by setting the Exchange's property with the key Exchange.CHARSET_NAME, or setting the encoding property on Xstream from DSL or Spring config.

from("activemq:My.Queue"). marshal().xstream("UTF-8"). to("mqseries:Another.Queue");

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-xstream/src/test/resources/org/apache/camel/dataformat/xstream/SpringMarshalListTest.xml}

Setting the type permissions of Xstream DataFormat

In Camel, one can always use its own processing step in the route to filter and block certain XML documents to be routed to the XStream's unmarhall step. From Camel 2.16.1, 2.15.5, you can set XStream's type permissions to automatically allow or deny the instantiation of certain types.

The default type permissions setting used by Camel denies all types except for those from java.lang and java.util packages. This setting can be changed by setting System property org.apache.camel.xstream.permissions. Its value is a string of comma-separated permission terms, each representing a type being allowed or denied, depending on whether the term is prefixed with '+' (note '+' may be omitted) or with '-', respectively.

Each term may contain a wildcard character '*'. For example, value "-*,java.lang.*,java.util.*" indicates denying all types except for java.lang.* and java.util.* classes. Setting this value to an empty string "" reverts to the default XStream's type permissions handling which denies certain blacklisted classes and allow others.

The type permissions setting can be extended at an individual XStream DataFormat instance by setting its type permissions property.

<dataFormats> <xstream id="xstream-default" permissions="org.apache.camel.samples.xstream.*"/> ...

CSV

The CSV Data Format uses Apache Commons CSV to handle CSV payloads (Comma Separated Values) such as those exported/imported by Excel.

As of Camel 2.15.0, it now uses the Apache Commons CSV 1.1 which is based on a completely different set of options.

Available options until Camel 2.15

Option

Type

Description

config

CSVConfig

Can be used to set a custom CSVConfig object.

strategy

CSVStrategy

Can be used to set a custom CSVStrategy; the default is CSVStrategy.DEFAULT_STRATEGY.

autogenColumns

boolean

Whether or not columns are auto-generated in the resulting CSV. The default value is true; subsequent messages use the previously created columns with new fields being added at the end of the line.

delimiter

String

Camel 2.4: The column delimiter to use; the default value is ",".

skipFirstLine

boolean

Camel 2.10: Whether or not to skip the first line of CSV input when unmarshalling (e.g. if the content has headers on the first line); the default value is false.

lazyLoadbooleanCamel 2.12.2: Whether or not to Sequential access CSV input through an iterator which could avoid OOM exception when processing huge CSV file; the default value is false
useMapsbooleanCamel 2.13: Whether to use List<Map> when unmarshalling instead of List<List>.

Available options as of Camel 2.15

OptionTypeDescription
formatCSVFormatThe reference format to use, it will be updated with the other format options, the default value is CSVFormat.DEFAULT
commentMarkerDisabledboolean

Disables the comment marker of the reference format.

This option is false by default.

commentMarkerCharacter

Overrides the comment marker of the reference format.

This option is null by default. When null it keeps the value of the reference format which is null for CSVFormat.DEFAULT.

delimiterCharacter

Overrides the delimiter of the reference format.

This option is null by defaut. When null it keeps the value of the reference format which is ',' for CSVFormat.DEFAULT.

escapeDisabledboolean

Disables the escape character of the reference format.

This option is false by default.

escapeCharacter

Overrides the escape character of the reference format.

This option is null by default. When null it keeps the value of the reference format which is null for CSVFormat.DEFAULT.

headerDisabledboolean

Disables the header of the reference format.

This option is false by default.

headerString[]

Overrides the header of the reference format.

This option is null by default. When null it keeps the value of the reference format which is null for CSVFormat.DEFAULT.

In the XML DSL, this option is configured using children <header> tags:

<csv >
    <header>orderId</header>
    <header>amount</header>
</csv>
allowMissingColumnNamesBoolean

Overrides the missing column names behavior of the reference format.

This option is null by default. When null it keeps the value of the reference format which is false for CSVFormat.DEFAULT.

ignoreEmptyLinesBoolean

Overrides the empty line behavior of the reference format.

This option is null by default. When null it keeps the value of the reference format which is true for CSVFormat.DEFAULT.

ignoreSurroundingSpacesBoolean

Overrides the surrounding spaces behavior of the reference format.

This option is null by default. When null it keeps the value of the reference format which is false for CSVFormat.DEFAULT.

nullStringDisabledboolean

Disables the null string representation of the reference format.

This option is false by default.

nullStringString

Overrides the null string representation of the reference format.

This option is null by default. When null it keeps the value of the reference format which is null for CSVFormat.DEFAULT.

quoteDisabledboolean

Disables the quote of the reference format.

This option is false by default.

quoteCharacter

Overrides the quote symbol of the reference format.

This option is null by default. When null it keeps the value of the reference format which is '"' (double quote) for CSVFormat.DEFAULT.

quoteModeQuoteMode

Overrides the quote mode of the reference format.

This option is null by default. When null it keeps the value of the reference format which is null for CSVFormat.DEFAULT.

recordSeparatorDisabledboolean

Disables the record separator of the reference format.

This option is false by default.

recordSeparatorString

Overrides the record separator of the reference format.

This option is null by default. When null it keeps the value of the reference format which is \r\n (CRLF) for CSVFormat.DEFAULT.

skipHeaderRecordBoolean

Overrides the header record behavior of the reference format.

This option is null by default. When null it keeps the value of the reference format which is false for CSVFormat.DEFAULT.

lazyLoadboolean

Whether the unmarshalling should produce an iterator that reads the lines on the fly or if all the lines must be read at one.

This option is false by default.

useMapsboolean

Whether the unmarshalling should produce maps for the lines values instead of lists. It requires to have header (either defined or collected).

This options is false by default.

recordConverterCsvRecordConverter

Sets the record converter to use. If defines the useMaps options is disabled.

This option is null by default.

Marshalling a Map to CSV

The component allows you to marshal a Java Map (or any other message type that can be converted in a Map) into a CSV payload.

Considering the following body
Map<String, Object> body = new LinkedHashMap<>();
body.put("foo", "abc");
body.put("bar", 123);
and this Java route definition
from("direct:start")
    .marshal().csv()
    .to("mock:result");
or this XML route definition
<route>
    <from uri="direct:start" />
    <marshal>
        <csv />
    </marshal>
    <to uri="mock:result" />
</route>
then it will produce
abc,123

Unmarshalling a CSV message into a Java List

Unmarshalling will transform a CSV messsage into a Java List with CSV file lines (containing another List with all the field values).

An example: we have a CSV file with names of persons, their IQ and their current activity.

Jack Dalton, 115, mad at Averell
Joe Dalton, 105, calming Joe
William Dalton, 105, keeping Joe from killing Averell
Averell Dalton, 80, playing with Rantanplan
Lucky Luke, 120, capturing the Daltons

We can now use the CSV component to unmarshal this file:

from("file:src/test/resources/?fileName=daltons.csv&noop=true")
    .unmarshal().csv()
    .to("mock:daltons");

The resulting message will contain a List<List<String>> like...

List<List<String>> data = (List<List<String>>) exchange.getIn().getBody();
for (List<String> line : data) {
    LOG.debug(String.format("%s has an IQ of %s and is currently %s", line.get(0), line.get(1), line.get(2)));
}

Marshalling a List<Map> to CSV

Available as of Camel 2.1

If you have multiple rows of data you want to be marshalled into CSV format you can now store the message payload as a List<Map<String, Object>> object where the list contains a Map for each row.

File Poller of CSV, then unmarshaling

Given a bean which can handle the incoming data...

MyCsvHandler.java
// Some comments here
public void doHandleCsvData(List<List<String>> csvData)
{
    // do magic here
}

... your route then looks as follows

<route>
        <!-- poll every 10 seconds -->
        <from uri="file:///some/path/to/pickup/csvfiles?delete=true&amp;consumer.delay=10000" />
        <unmarshal><csv /></unmarshal>
        <to uri="bean:myCsvHandler?method=doHandleCsvData" />
</route>

Marshaling with a pipe as delimiter

 

 

Considering the following body
Map<String, Object> body = new LinkedHashMap<>();
body.put("foo", "abc");
body.put("bar", 123);
and this Java route definition
// Camel version < 2.15
CsvDataFormat oldCSV = new CsvDataFormat();
oldCSV.setDelimiter("|");
from("direct:start")
    .marshal(oldCSV)
    .to("mock:result")
 
// Camel version >= 2.15
from("direct:start")
    .marshal(new CsvDataFormat().setDelimiter('|'))
    .to("mock:result")
or this XML route definition
<route>
  <from uri="direct:start" />
  <marshal>
    <csv delimiter="|" />
  </marshal>
  <to uri="mock:result" />
</route>
then it will produce
abc|123

Using autogenColumns, configRef and strategyRef attributes inside XML DSL

Available as of Camel 2.9.2 / 2.10 and deleted for Camel 2.15

You can customize the CSV Data Format to make use of your own CSVConfig and/or CSVStrategy. Also note that the default value of the autogenColumns option is true. The following example should illustrate this customization.

<route>
  <from uri="direct:start" />
  <marshal>
    <!-- make use of a strategy other than the default one which is 'org.apache.commons.csv.CSVStrategy.DEFAULT_STRATEGY' -->
    <csv autogenColumns="false" delimiter="|" configRef="csvConfig" strategyRef="excelStrategy" />
  </marshal>
  <convertBodyTo type="java.lang.String" />
  <to uri="mock:result" />
</route>

<bean id="csvConfig" class="org.apache.commons.csv.writer.CSVConfig">
  <property name="fields">
    <list>
      <bean class="org.apache.commons.csv.writer.CSVField">
        <property name="name" value="orderId" />
      </bean>
      <bean class="org.apache.commons.csv.writer.CSVField">
        <property name="name" value="amount" />
      </bean>
    </list>
  </property>
</bean>

<bean id="excelStrategy" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean">
  <property name="staticField" value="org.apache.commons.csv.CSVStrategy.EXCEL_STRATEGY" />
</bean>

Using skipFirstLine option while unmarshaling

Available as of Camel 2.10 and deleted for Camel 2.15

You can instruct the CSV Data Format to skip the first line which contains the CSV headers. Using the Spring/XML DSL:

<route>
  <from uri="direct:start" />
  <unmarshal>
    <csv skipFirstLine="true" />
  </unmarshal>
  <to uri="bean:myCsvHandler?method=doHandleCsv" />
</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();
csv.setSkipFirstLine(true);

from("direct:start")
  .unmarshal(csv)
.to("bean:myCsvHandler?method=doHandleCsv");

Unmarshaling with a pipe as delimiter

Using the Spring/XML DSL:

<route>
  <from uri="direct:start" />
  <unmarshal>
    <csv delimiter="|" />
  </unmarshal>
  <to uri="bean:myCsvHandler?method=doHandleCsv" />
</route>

Or the Java DSL:

CsvDataFormat csv = new CsvDataFormat();
CSVStrategy strategy = CSVStrategy.DEFAULT_STRATEGY;
strategy.setDelimiter('|');
csv.setStrategy(strategy);

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");
CsvDataFormat csv = new CsvDataFormat();
csv.setDelimiter("|");

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");
CsvDataFormat csv = new CsvDataFormat();
CSVConfig csvConfig = new CSVConfig();
csvConfig.setDelimiter(";");
csv.setConfig(csvConfig);

from("direct:start")
  .unmarshal(csv)
  .to("bean:myCsvHandler?method=doHandleCsv");

Issue in CSVConfig

It looks like that

CSVConfig csvConfig = new CSVConfig();
csvConfig.setDelimiter(';');

doesn't work. You have to set the delimiter as a String!

Dependencies

To use CSV in your Camel routes you need to add a dependency on camel-csv, which implements this data format.

If you use Maven you can just add the following to your pom.xml, substituting the version number for the latest and greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-csv</artifactId>
  <version>x.x.x</version>
</dependency>
The String Data Format is a textual based format that supports encoding.

Options

Option

Default

Description

charset

null

To use a specific charset for encoding. If not provided Camel will use the JVM default charset.

Marshal

In this example we marshal the file content to String object in UTF-8 encoding.

from("file://data.csv").marshal().string("UTF-8").to("jms://myqueue");

Unmarshal

In this example we unmarshal the payload from the JMS queue to a String object using UTF-8 encoding, before its processed by the newOrder processor.

from("jms://queue/order").unmarshal().string("UTF-8").processRef("newOrder");

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

HL7 DataFormat

The HL7 component ships with a HL7 data format that can be used to marshal or unmarshal HL7 model objects.

  • marshal = from Message to byte stream (can be used when responding using the HL7 MLLP codec)
  • unmarshal = from byte stream to Message (can be used when receiving streamed data from the HL7 MLLP

To use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder:

  DataFormat hl7 = new HL7DataFormat();
  ...
  from("direct:hl7in").marshal(hl7).to("jms:queue:hl7out");

In the sample above, the HL7 is marshalled from a HAPI Message object to a byte stream and put on a JMS queue.
The next example is the opposite:

  DataFormat hl7 = new HL7DataFormat();
  ...
  from("jms:queue:hl7out").unmarshal(hl7).to("patientLookupService");

Here we unmarshal the byte stream into a HAPI Message object that is passed to our patient lookup service.

Serializable messages

As of HAPI 2.0 (used by Camel 2.11), the HL7v2 model classes are fully serializable. So you can put HL7v2 messages directly into a JMS queue (i.e. without calling marshal() and read them again directly from the queue (i.e. without calling unmarshal().

Segment separators

As of Camel 2.11, unmarshal does not automatically fix segment separators anymore by converting \n to \r. If you
need this conversion, org.apache.camel.component.hl7.HL7#convertLFToCR provides a handy Expression for this purpose.

Charset

As of Camel 2.14.1, both marshal and unmarshal evaluate the charset provided in the field MSH-18. If this field is empty, by default the charset contained in the corresponding Camel charset property/header is assumed. You can even change this default behavior by overriding the guessCharsetName method when inheriting from the HL7DataFormat class.

 

There is a shorthand syntax in Camel for well-known data formats that are commonly used.
Then you don't need to create an instance of the HL7DataFormat object:

  from("direct:hl7in").marshal().hl7().to("jms:queue:hl7out");
  from("jms:queue:hl7out").unmarshal().hl7().to("patientLookupService");

 

 

EDI DataFormat

We encourage end users to look at the Smooks which supports EDI and Camel natively.

Flatpack DataFormat

The Flatpack component ships with the Flatpack data format that can be used to format between fixed width or delimited text messages to a List of rows as Map.

  • marshal = from List<Map<String, Object>> to OutputStream (can be converted to String)
  • unmarshal = from java.io.InputStream (such as a File or String) to a java.util.List as an org.apache.camel.component.flatpack.DataSetList instance.
    The result of the operation will contain all the data. If you need to process each row one by one you can split the exchange, using Splitter.

Notice: The Flatpack library does currently not support header and trailers for the marshal operation.

Options

The data format has the following options:

Option

Default

Description

definition

null

The flatpack pzmap configuration file. Can be omitted in simpler situations, but its preferred to use the pzmap.

fixed

false

Delimited or fixed.

ignoreFirstRecord

true

Whether the first line is ignored for delimited files (for the column headers).

textQualifier

"

If the text is qualified with a char such as ".

delimiter

,

The delimiter char (could be ; , or similar)

parserFactory

null

Uses the default Flatpack parser factory.

allowShortLines

false

Camel 2.9.7 and 2.10.5 onwards: Allows for lines to be shorter than expected and ignores the extra characters.

ignoreExtraColumns

false

Camel 2.9.7 and 2.10.5 onwards: Allows for lines to be longer than expected and ignores the extra characters.

Usage

To use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder:

  FlatpackDataFormat fp = new FlatpackDataFormat();
  fp.setDefinition(new ClassPathResource("INVENTORY-Delimited.pzmap.xml"));
  ...
  from("file:order/in").unmarshal(df).to("seda:queue:neworder");

The sample above will read files from the order/in folder and unmarshal the input using the Flatpack configuration file INVENTORY-Delimited.pzmap.xml that configures the structure of the files. The result is a DataSetList object we store on the SEDA queue.

FlatpackDataFormat df = new FlatpackDataFormat();
df.setDefinition(new ClassPathResource("PEOPLE-FixedLength.pzmap.xml"));
df.setFixed(true);
df.setIgnoreFirstRecord(false);

from("seda:people").marshal(df).convertBodyTo(String.class).to("jms:queue:people");

In the code above we marshal the data from a Object representation as a List of rows as Maps. The rows as Map contains the column name as the key, and the the corresponding value. This structure can be created in Java code from e.g. a processor. We marshal the data according to the Flatpack format and convert the result as a String object and store it on a JMS queue.

Dependencies

To use Flatpack in your camel routes you need to add the a dependency on camel-flatpack which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-flatpack</artifactId>
  <version>x.x.x</version>
</dependency>

JSON

JSON is a Data Format to marshal and unmarshal Java objects to and from JSON.

For JSON to object marshalling, Camel provides integration with three popular JSON libraries:

Every library requires adding the special camel component (see "Dependency..." paragraphs further down). By default Camel uses the XStream library.

Direct, bi-directional JSON <=> XML conversions

As of Camel 2.10, Camel supports direct, bi-directional JSON <=> XML conversions via the camel-xmljson data format, which is documented separately.

Using JSON Data Format With the XStream Library

java// Let's turn Object messages into JSON then send to MQSeries from("activemq:My.Queue") .marshal().json() .to("mqseries:Another.Queue");

Using JSON Data Format With the Jackson Library

java// Let's turn Object messages into JSON then send to MQSeries from("activemq:My.Queue") .marshal().json(JsonLibrary.Jackson) .to("mqseries:Another.Queue");

Using JSON Data Format With the GSON Library

java// Let's turn Object messages into JSON then send to MQSeries from("activemq:My.Queue") .marshal().json(JsonLibrary.Gson) .to("mqseries:Another.Queue");

Using JSON in Spring DSL

When using Data Format in Spring DSL you need to declare the data formats first. This is done in the DataFormats XML tag.

xml<dataFormats> <!-- Here we define a Json data format with the id jack and that it should use the TestPojo as the class type when doing unmarshal. The unmarshalTypeName is optional, if not provided Camel will use a Map as the type. --> <json id="jack" library="Jackson" unmarshalTypeName="org.apache.camel.component.jackson.TestPojo"/> </dataFormats>

And then you can refer to this id in the route:

xml<route> <from uri="direct:back"/> <unmarshal ref="jack"/> <to uri="mock:reverse"/> </route>

Excluding POJO Fields From Marshalling

As of Camel 2.10
When marshaling a POJO to JSON you might want to exclude certain fields from the JSON output. With Jackson you can use JSON views to accomplish this.

First create one or more marker classes:{snippet:id=marker|lang=java|url=camel/trunk/components/camel-jackson/src/test/java/org/apache/camel/component/jackson/Views.java}Second, use the marker classes with the @JsonView annotation to include/exclude certain fields. The annotation also works on getters:{snippet:id=jsonview|lang=java|url=camel/trunk/components/camel-jackson/src/test/java/org/apache/camel/component/jackson/TestPojoView.java}Finally, use the Camel JacksonDataFormat to marshal the above POJO to JSON.{snippet:id=format|lang=java|url=camel/trunk/components/camel-jackson/src/test/java/org/apache/camel/component/jackson/JacksonMarshalViewTest.java}Note: the height field is missing in the resulting JSON.

{"age":30, "weight":70}

The GSON library supports a similar feature through the notion of ExclusionStrategies:{snippet:id=strategy|lang=java|url=camel/trunk/components/camel-gson/src/test/java/org/apache/camel/component/gson/GsonMarshalExclusionTest.java}The GsonDataFormat accepts an ExclusionStrategy in its constructor:{snippet:id=format|lang=java|url=camel/trunk/components/camel-gson/src/test/java/org/apache/camel/component/gson/GsonMarshalExclusionTest.java}The line above will exclude fields annotated with @ExcludeAge when marshaling to JSON.

Configuring Field Naming Policy

Available as of Camel 2.11

The GSON library supports specifying policies and strategies for mapping from JSON to POJO fields. A common naming convention is to map JSON fields using lower case with underscores.

We may have this JSON string

{ "id" : 123, "first_name" : "Donald" "last_name" : "Duck" }

Which we want to map to a POJO that has getter/setters as:

javaPersonPojo.javaPersonPojo.javapublic class PersonPojo { private int id; private String firstName; private String lastName; public int getId() { return id; } public void setId(int id) { this.id = id; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } }

Then we can configure the org.apache.camel.component.gson.GsonDataFormat in a Spring XML files as shown below. Notice we use fieldNamingPolicy property to set the field mapping. This property is an enum from GSon com.google.gson.FieldNamingPolicy which has a number of predefined mappings.

If you need full control you can use the property FieldNamingStrategy and implement a custom com.google.gson.FieldNamingStrategy where you can control the mapping.

Configuring GsonDataFormat in Spring XML filexmlConfiguring GsonDataFromat in Spring XML file<!-- define the gson data format, where we configure the data format using the properties --> <bean id="gson" class="org.apache.camel.component.gson.GsonDataFormat"> <!-- we want to unmarshal to person pojo --> <property name="unmarshalType" value="org.apache.camel.component.gson.PersonPojo"/> <!-- we want to map fields to use lower case and underscores --> <property name="fieldNamingPolicy" value="LOWER_CASE_WITH_UNDERSCORES"/> </bean>

And use it in Camel routes by referring to its bean id as shown:

Using gson from Camel routesxmlUsing gson from Camel Routes<camelContext xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:inPojo"/> <marshal ref="gson"/> </route> <route> <from uri="direct:backPojo"/> <unmarshal ref="gson"/> </route> </camelContext>

Include/Exclude Fields Using the jsonView Attribute With JacksonDataFormat

Available as of Camel 2.12

As an example of using this attribute you can instead of:

javaJacksonDataFormat ageViewFormat = new JacksonDataFormat(TestPojoView.class, Views.Age.class); from("direct:inPojoAgeView") .marshal(ageViewFormat);

Directly specify your JSON view inside the Java DSL as:

javafrom("direct:inPojoAgeView") .marshal().json(TestPojoView.class, Views.Age.class);

And the same in XML DSL:

xml<from uri="direct:inPojoAgeView"/> <marshal> <json library="Jackson" unmarshalTypeName="org.apache.camel.component.jackson.TestPojoView" jsonView="org.apache.camel.component.jackson.Views$Age"/> </marshal>

Setting Serialization Include Option for Jackson Marshal

Available as of Camel 2.13.3/2.14

If you want to marshal a POJO to JSON, and the POJO has some fields with null values. And you want to skip these null values, then you need to set either an annotation on the POJO, 

java@JsonInclude(Include.NON_NULL) public class MyPojo { // ... }

But this requires you to include that annotation in your POJO source code. You can also configure the Camel JsonDataFormat to set the include option, as shown below:

javaJacksonDataFormat format = new JacksonDataFormat(); format.setInclude("NON_NULL");

Or from XML DSL you configure this as

xml<dataFormats> <json id="json" library="Jackson" include="NON_NULL"/> </dataFormats>

Unmarshaling from JSON to POJO with Dynamic Class Name

Available as of Camel 2.14

If you use Jackson to unmarshal JSON to POJO, then you can now specify a header in the message that indicate which class name to unmarshal to. The header has key CamelJacksonUnmarshalType if that header is present in the message, then Jackson will use that as FQN for the POJO class to unmarshal the JSON payload as. Notice that behavior is enabled out of the box from Camel 2.14

 For JMS end users there is the JMSType header from the JMS spec that indicates that also. To enable support for JMSType you would need to turn that on, on the Jackson data format as shown:

javaJacksonDataFormat format = new JacksonDataFormat(); format.setAllowJmsType(true);

Or from XML DSL you configure this as

xml<dataFormats> <json id="json" library="Jackson" allowJmsType="true"/> </dataFormats>

Unmarshaling From JSON to List<Map> or List<pojo>

Available as of Camel 2.14

If you are using Jackson to unmarshal JSON to a list of map/POJO, you can now specify this by setting useList="true" or use the org.apache.camel.component.jackson.ListJacksonDataFormat.

For example, with Java you can do as shown below:

javaJacksonDataFormat format = new ListJacksonDataFormat(); // or JacksonDataFormat format = new JacksonDataFormat(); format.useList(); // and you can specify the pojo class type also format.setUnmarshalType(MyPojo.class);

And if you use XML DSL then you configure to use list using useList attribute as shown below:

xml<dataFormats> <json id="json" library="Jackson" useList="true"/> </dataFormats>

And you can specify the pojo type also

xml<dataFormats> <json id="json" library="Jackson" useList="true" unmarshalTypeName="com.foo.MyPojo"/> </dataFormats>

Using Custom Jackson's ObjectMapper

Available from Camel 2.17

You can use custom Jackson ObjectMapper instance, can be configured as shown below.

xml<dataFormats> <json id="json" library="Jackson" objectMapper="myMapper"/> </dataFormats>

Where myMapper is the id of the custom instance that Camel will lookup in the Registry.

Using Custom Jackson Modules

Available as of Camel 2.15

You can use custom Jackson modules by specifying the class names of those using the moduleClassNames option as shown below.

xml<dataFormats> <json id="json" library="Jackson" useList="true" unmarshalTypeName="com.foo.MyPojo" moduleClassNames="com.foo.MyModule,com.foo.MyOtherModule"/> </dataFormats>

When using moduleClassNames then the custom Jackson modules are not configured, by created using default constructor and used as-is. If a custom module needs any custom configuration, then an instance of the module can be created and configured, and then use modulesRefs to refer to the module as shown below:

xml<bean id="myJacksonModule" class="com.foo.MyModule"> ... // configure the module as you want </bean>   <dataFormats> <json id="json" library="Jackson" useList="true" unmarshalTypeName="com.foo.MyPojo" moduleRefs="myJacksonModule"/> </dataFormats>

 Multiple modules can be specified separated by comma, such as moduleRefs="myJacksonModule,myOtherModule".

Enabling or Disable Features Using Jackson

Available as of Camel 2.15

Jackson has a number of features you can enable or disable, which its ObjectMapper uses. For example to disable failing on unknown properties when marshaling, you can configure this using the disableFeatures:

xml<dataFormats> <json id="json" library="Jackson" unmarshalTypeName="com.foo.MyPojo" disableFeatures="FAIL_ON_UNKNOWN_PROPERTIES"/> </dataFormats>

You can disable multiple features by separating the values using comma. The values for the features must be the name of the enums from Jackson from the following enum classes

  • com.fasterxml.jackson.databind.SerializationFeature
  • com.fasterxml.jackson.databind.DeserializationFeature
  • com.fasterxml.jackson.databind.MapperFeature

To enable a feature use the enableFeatures options instead.

From Java code you can use the type safe methods from camel-jackson module:

javaJacksonDataFormat df = new JacksonDataFormat(MyPojo.class); df.disableFeature(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); df.disableFeature(DeserializationFeature.FAIL_ON_NULL_FOR_PRIMITIVES);

Converting Maps to POJO Using Jackson

Available since Camel 2.16. Jackson ObjectMapper can be used to convert maps to POJO objects. Jackson component comes with the data converter that can be used to convert java.util.Map instance to non-String, non-primitive and non-Number objects.

javaMap<String, Object> invoiceData = new HashMap<String, Object>(); invoiceData.put("netValue", 500); producerTemplate.sendBody("direct:mapToInvoice", invoiceData); ... // Later in the processor Invoice invoice = exchange.getIn().getBody(Invoice.class);

If there is a single ObjectMapper instance available in the Camel registry, it will used by the converter to perform the conversion. Otherwise the default mapper will be used.  

Formatted JSON marshalling (pretty-printing)

Available as of Camel 2.16

Using the prettyPrint option one can output a well formatted JSON while marshaling:

xml<dataFormats> <json id="xstream" prettyPrint="true"/> <json id="jackson" prettyPrint="true" library="Jackson"/> <json id="gson" prettyPrint="true" library="Gson"/> </dataFormats>

And in Java DSL:

javafrom("direct:inPretty").marshal().json(true);   from("direct:inPretty").marshal().json(JsonLibrary.Jackson, true);   from("direct:inPretty").marshal().json(JsonLibrary.Gson, true);

Please note that as of Camel 2.16 there are five different overloaded json() DSL methods which support the prettyPrint option in combination with other settings for JsonLibrary, unmarshalType, jsonView etc. 

 Integrating Jackson with Camel's TypeConverters

 Available as of Camel 2.17

The camel-jackson module allows to integrate Jackson as a Type Converter in the Camel registry. This works in similar ways that camel-jaxb integrates with the type converter as well. However camel-jackson must be explicit enabled, which is done by setting some options on the CamelContext properties, as shown below:

java// Enable Jackson JSON type converter. getContext().getProperties().put("CamelJacksonEnableTypeConverter", "true"); // Allow Jackson JSON to convert to pojo types also (by default Jackson only converts to String and other simple types). getContext().getProperties().put("CamelJacksonTypeConverterToPojo", "true");

The camel-jackson type converter integrates with JAXB which means you can annotate POJO class with JAXB annotations that Jackson can leverage. 

Dependencies for XStream

To use JSON in your camel routes you need to add the a dependency on camel-xstream which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

xml<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>2.9.2</version> </dependency>

Dependencies for Jackson

To use JSON in your camel routes you need to add the a dependency on camel-jackson which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

xml<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jackson</artifactId> <version>2.9.2</version> </dependency>

Dependencies for GSON

To use JSON in your camel routes you need to add the a dependency on camel-gson which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

xml<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-gson</artifactId> <version>2.10.0</version> </dependency>
The Zip Data Format is a message compression and de-compression format. Messages marshalled using Zip compression can be unmarshalled using Zip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads. It facilitates more optimal use of network bandwidth while incurring a small cost in order to compress and decompress payloads at the endpoint.

About using with Files

The Zip data format, does not (yet) have special support for files. Which means that when using big files, the entire file content is loaded into memory.
This is subject to change in the future, to allow a streaming based solution to have a low memory footprint.

Options

Option

Default

Description

compressionLevel

null

To specify a specific compression Level use java.util.zip.Deflater settings. The possible settings are 
          - Deflater.BEST_SPEED
          - Deflater.BEST_COMPRESSION
          - Deflater.DEFAULT_COMPRESSION

If compressionLevel is not explicitly specified the compressionLevel employed is Deflater.DEFAULT_COMPRESSION

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing zip compression Deflater.BEST_COMPRESSION and send it an ActiveMQ queue called MY_QUEUE.

from("direct:start").marshal().zip(Deflater.BEST_COMPRESSION).to("activemq:queue:MY_QUEUE");

Alternatively if you would like to use the default setting you could send it as

from("direct:start").marshal().zip().to("activemq:queue:MY_QUEUE");

Unmarshal

In this example we unmarshal a zipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnZippedMessageProcessor. Note that the compression Level employed during the marshalling should be identical to the one employed during unmarshalling to avoid errors.

from("activemq:queue:MY_QUEUE").unmarshal().zip().process(new UnZippedMessageProcessor()); 

Dependencies

This data format is provided in camel-core so no additional dependencies are needed.

TidyMarkup

TidyMarkup is a Data Format that uses the TagSoup to tidy up HTML. It can be used to parse ugly HTML and return it as pretty wellformed HTML.

Camel eats our own -dog food- soap

We had some issues in our pdf Manual where we had some strange symbols. So Jonathan used this data format to tidy up the wiki html pages that are used as base for rendering the pdf manuals. And then the mysterious symbols vanished.

TidyMarkup only supports the unmarshal operation as we really don't want to turn well formed HTML into ugly HTML (smile)

Java DSL Example

An example where the consumer provides some HTML

from("file://site/inbox").unmarshal().tidyMarkup().to("file://site/blogs");

Spring XML Example

The following example shows how to use TidyMarkup to unmarshal using Spring

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
  <route>
    <from uri="file://site/inbox"/>
    <unmarshal>
      <tidyMarkup/>
    </unmarshal>
    <to uri="file://site/blogs"/>
  </route>
</camelContext>

Dependencies

To use TidyMarkup in your camel routes you need to add the a dependency on camel-tagsoup which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-tagsoup</artifactId>
  <version>x.x.x</version>
</dependency>

Bindy

The goal of this component is to allow the parsing/binding of non-structured data (or to be more precise non-XML data)
to/from Java Beans that have binding mappings defined with annotations. Using Bindy, you can bind data from sources such as :

  • CSV records,
  • Fixed-length records,
  • FIX messages,
  • or almost any other non-structured data

to one or many Plain Old Java Object (POJO). Bindy converts the data according to the type of the java property. POJOs can be linked together with one-to-many relationships available in some cases. Moreover, for data type like Date, Double, Float, Integer, Short, Long and BigDecimal, you can provide the pattern to apply during the formatting of the property.

For the BigDecimal numbers, you can also define the precision and the decimal or grouping separators.

Type

Format Type

Pattern example

Link

Date

DateFormat

"dd-MM-yyyy"

http://java.sun.com/j2se/1.5.0/docs/api/java/text/SimpleDateFormat.html

Decimal*

Decimalformat

"##.###.###"

http://java.sun.com/j2se/1.5.0/docs/api/java/text/DecimalFormat.html

Decimal* = Double, Integer, Float, Short, Long

Format supported

This first release only support comma separated values fields and key value pair fields (e.g. : FIX messages).

To work with camel-bindy, you must first define your model in a package (e.g. com.acme.model) and for each model class (e.g. Order, Client, Instrument, ...) add the required annotations (described hereafter) to the Class or field.

Multiple models

If you use multiple models, each model has to be placed in it's own package to prevent unpredictable results.

From Camel 2.16 onwards this is no longer the case, as you can safely have multiple models in the same package, as you configure bindy using class names instead of package names now.

Annotations

The annotations created allow to map different concept of your model to the POJO like :

  • Type of record (csv, key value pair (e.g. FIX message), fixed length ...),
  • Link (to link object in another object),
  • DataField and their properties (int, type, ...),
  • KeyValuePairField (for key = value format like we have in FIX financial messages),
  • Section (to identify header, body and footer section),
  • OneToMany

This section will describe them :

1. CsvRecord

The CsvRecord annotation is used to identified the root class of the model. It represents a record = a line of a CSV file and can be linked to several children model classes.

Annotation name

Record type

Level

CsvRecord

csv

Class

Parameter name

type

Info

separator

string

mandatory - can be ',' or ';' or 'anything'. This value is interpreted as a regular expression. If you want to use a sign which has a special meaning in regular expressions, e.g. the '|' sign, than you have to mask it, like '
|'

skipFirstLine

boolean

optional - default value = false - allow to skip the first line of the CSV file

crlf

string

optional - possible values = WINDOWS,UNIX,MAC, or custom; default value = WINDOWS - allow to define the carriage return character to use. If you specify a value other than the three listed before, the value you enter (custom) will be used as the CRLF character(s)

generateHeaderColumns

boolean

optional - default value = false - uses to generate the header columns of the CSV generates

autospanLine

boolean

Camel 2.13/2.12.2: optional - default value = false - if enabled then the last column is auto spanned to end of line, for example if its a comment, etc this allows the line to contain all characters, also the delimiter char.

isOrdered

boolean

optional - default value = false - allow to change the order of the fields when CSV is generated

quote

String

Camel 2.8.3/2.9: option - allow to specify a quote character of the fields when CSV is generated

 

 

This annotation is associated to the root class of the model and must be declared one time.

quotingbooleanCamel 2.11:optional - default value = false - Indicate if the values must be quoted when marshaling when CSV is generated.

case 1 : separator = ','

The separator used to segregate the fields in the CSV record is ',' :

10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

@CsvRecord( separator = "," )
public Class Order {
...
}

case 2 : separator = ';'

Compare to the previous case, the separator here is ';' instead of ',' :

10; J; Pauline; M; XD12345678; Fortis Dynamic 15/15; 2500; USD; 08-01-2009

@CsvRecord( separator = ";" )
public Class Order {
...
}

case 3 : separator = '|'

Compare to the previous case, the separator here is '|' instead of ';' :

10| J| Pauline| M| XD12345678| Fortis Dynamic 15/15| 2500| USD| 08-01-2009

@CsvRecord( separator = "\\|" )
public Class Order {
...
}

case 4 : separator = '\",\"'
Applies for Camel 2.8.2 or older

When the field to be parsed of the CSV record contains ',' or ';' which is also used as separator, we whould find another strategy
to tell camel bindy how to handle this case. To define the field containing the data with a comma, you will use simple or double quotes
as delimiter (e.g : '10', 'Street 10, NY', 'USA' or "10", "Street 10, NY", "USA").
Remark : In this case, the first and last character of the line which are a simple or double quotes will removed by bindy

"10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009"

@CsvRecord( separator = "\",\"" )
public Class Order {
...
}

From Camel 2.8.3/2.9 or never bindy will automatic detect if the record is enclosed with either single or double quotes and automatic remove those quotes when unmarshalling from CSV to Object. Therefore do not include the quotes in the separator, but simple do as below:

"10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009"

@CsvRecord( separator = "," )
public Class Order {
...
}

Notice that if you want to marshal from Object to CSV and use quotes, then you need to specify which quote character to use, using the quote attribute on the @CsvRecord as shown below:

@CsvRecord( separator = ",", quote = "\"" )
public Class Order {
...
}

case 5 : separator & skipfirstline

The feature is interesting when the client wants to have in the first line of the file, the name of the data fields :

order id, client id, first name, last name, isin code, instrument name, quantity, currency, date

To inform bindy that this first line must be skipped during the parsing process, then we use the attribute :

@CsvRecord(separator = ",", skipFirstLine = true)
public Class Order {
...
}

case 6 : generateHeaderColumns

To add at the first line of the CSV generated, the attribute generateHeaderColumns must be set to true in the annotation like this :

@CsvRecord( generateHeaderColumns = true )
public Class Order {
...
}

As a result, Bindy during the unmarshaling process will generate CSV like this :

order id, client id, first name, last name, isin code, instrument name, quantity, currency, date
10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

case 7 : carriage return

If the platform where camel-bindy will run is not Windows but Macintosh or Unix, than you can change the crlf property like this. Three values are available : WINDOWS, UNIX or MAC

@CsvRecord(separator = ",", crlf="MAC")
public Class Order {
...
}

Additionally, if for some reason you need to add a different line ending character, you can opt to specify it using the crlf parameter. In the following example, we can end the line with a comma followed by the newline character:

@CsvRecord(separator = ",", crlf=",\n")
public Class Order {
...
}

case 8 : isOrdered

Sometimes, the order to follow during the creation of the CSV record from the model is different from the order used during the parsing. Then, in this case, we can use the attribute isOrdered = true to indicate this in combination with attribute 'position' of the DataField annotation.

@CsvRecord(isOrdered = true)
public Class Order {

   @DataField(pos = 1, position = 11)
   private int orderNr;

   @DataField(pos = 2, position = 10)
   private String clientNr;

...
}

Remark : pos is used to parse the file, stream while positions is used to generate the CSV

The link annotation will allow to link objects together.

Annotation name

Record type

Level

Link

all

Class & Property

Parameter name

type

Info

linkType

LinkType

optional - by default the value is LinkType.oneToOne - so you are not obliged to mention it

 

 

Only one-to-one relation is allowed.

e.g : If the model Class Client is linked to the Order class, then use annotation Link in the Order class like this :

Property Link
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @Link
    private Client client;
...

AND for the class Client :

Class Link
@Link
public class Client {
...
}

3. DataField

The DataField annotation defines the property of the field. Each datafield is identified by its position in the record, a type (string, int, date, ...) and optionally of a pattern

Annotation name

Record type

Level

DataField

all

Property

Parameter name

type

Info

pos

int

mandatory - The input position of the field. digit number starting from 1 to ... - See the position parameter.

pattern

string

optional - default value = "" - will be used to format Decimal, Date, ...

length

int

optional - represents the length of the field for fixed length format

precision

int

optional - represents the precision to be used when the Decimal number will be formatted/parsed

pattern

string

optional - default value = "" - is used by the Java formatter (SimpleDateFormat by example) to format/validate data. If using pattern, then setting locale on bindy data format is recommended. Either set to a known locale such as "us" or use "default" to use platform default locale. Notice that "default" requires Camel 2.14/2.13.3/2.12.5.

position

int

optional - must be used when the position of the field in the CSV generated (output message) must be different compare to input position (pos). See the pos parameter.

required

boolean

optional - default value = "false"

trim

boolean

optional - default value = "false"

defaultValue

string

Camel 2.10: optional - default value = "" - defines the field's default value when the respective CSV field is empty/not available

impliedDecimalSeparator

boolean

Camel 2.11: optional - default value = "false" - Indicates if there is a decimal point implied at a specified location

lengthPos

int

Camel 2.11: optional - can be used to identify a data field in a fixed-length record that defines the fixed length for this field

delimiter

string

Camel 2.11: optional - can be used to demarcate the end of a variable-length field within a fixed-length record

case 1 : pos

This parameter/attribute represents the position of the field in the csv record

Position
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @DataField(pos = 5)
    private String isinCode;

...
}

As you can see in this example the position starts at '1' but continues at '5' in the class Order. The numbers from '2' to '4' are defined in the class Client (see here after).

Position continues in another model class
public class Client {

    @DataField(pos = 2)
    private String clientNr;

    @DataField(pos = 3)
    private String firstName;

    @DataField(pos = 4)
    private String lastName;
...
}

case 2 : pattern

The pattern allows to enrich or validates the format of your data

Pattern
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @DataField(pos = 5)
    private String isinCode;

    @DataField(name = "Name", pos = 6)
    private String instrumentName;

    @DataField(pos = 7, precision = 2)
    private BigDecimal amount;

    @DataField(pos = 8)
    private String currency;

    @DataField(pos = 9, pattern = "dd-MM-yyyy") -- pattern used during parsing or when the date is created
    private Date orderDate;
...
}

case 3 : precision

The precision is helpful when you want to define the decimal part of your number

Precision
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @Link
    private Client client;

    @DataField(pos = 5)
    private String isinCode;

    @DataField(name = "Name", pos = 6)
    private String instrumentName;

    @DataField(pos = 7, precision = 2) -- precision
    private BigDecimal amount;

    @DataField(pos = 8)
    private String currency;

    @DataField(pos = 9, pattern = "dd-MM-yyyy")
    private Date orderDate;
...
}

case 4 : Position is different in output

The position attribute will inform bindy how to place the field in the CSV record generated. By default, the position used corresponds to the position defined with the attribute 'pos'. If the position is different (that means that we have an asymetric processus comparing marshaling from unmarshaling) than we can use 'position' to indicate this.

Here is an example

Position is different in output
@CsvRecord(separator = ",")
public class Order {
@CsvRecord(separator = ",", isOrdered = true)
public class Order {

    // Positions of the fields start from 1 and not from 0

    @DataField(pos = 1, position = 11)
    private int orderNr;

    @DataField(pos = 2, position = 10)
    private String clientNr;

    @DataField(pos = 3, position = 9)
    private String firstName;

    @DataField(pos = 4, position = 8)
    private String lastName;

    @DataField(pos = 5, position = 7)
    private String instrumentCode;

    @DataField(pos = 6, position = 6)
    private String instrumentNumber;
...
}

This attribute of the annotation @DataField must be used in combination with attribute isOrdered = true of the annotation @CsvRecord

case 5 : required

If a field is mandatory, simply use the attribute 'required' setted to true

Required
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @DataField(pos = 2, required = true)
    private String clientNr;

    @DataField(pos = 3, required = true)
    private String firstName;

    @DataField(pos = 4, required = true)
    private String lastName;
...
}

If this field is not present in the record, than an error will be raised by the parser with the following information :

Some fields are missing (optional or mandatory), line :

case 6 : trim

If a field has leading and/or trailing spaces which should be removed before they are processed, simply use the attribute 'trim' setted to true

Trim
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1, trim = true)
    private int orderNr;

    @DataField(pos = 2, trim = true)
    private Integer clientNr;

    @DataField(pos = 3, required = true)
    private String firstName;

    @DataField(pos = 4)
    private String lastName;
...
}

case 7 : defaultValue

If a field is not defined then uses the value indicated by the defaultValue attribute

Default value
@CsvRecord(separator = ",")
public class Order {

    @DataField(pos = 1)
    private int orderNr;

    @DataField(pos = 2)
    private Integer clientNr;

    @DataField(pos = 3, required = true)
    private String firstName;

    @DataField(pos = 4, defaultValue = "Barin")
    private String lastName;
...
}

This attribute is only applicable to optional fields.

4. FixedLengthRecord

The FixedLengthRecord annotation is used to identified the root class of the model. It represents a record = a line of a file/message containing data fixed length formatted and can be linked to several children model classes. This format is a bit particular beause data of a field can be aligned to the right or to the left.
When the size of the data does not fill completely the length of the field, we can then add 'padd' characters.

Annotation name

Record type

Level

FixedLengthRecord

fixed

Class

Parameter name

type

Info

crlf

string

optional - possible values = WINDOWS,UNIX,MAC, or custom; default value = WINDOWS - allow to define the carriage return character to use. If you specify a value other than the three listed before, the value you enter (custom) will be used as the CRLF character(s)

paddingChar

char

mandatory - default value = ' '

length

int

mandatory = size of the fixed length record

hasHeader

boolean

Camel 2.11 - optional - Indicates that the record(s) of this type may be preceded by a single header record at the beginning of the file / stream

hasFooter

boolean

Camel 2.11 - optional - Indicates that the record(s) of this type may be followed by a single footer record at the end of the file / stream

skipHeader

boolean

Camel 2.11 - optional - Configures the data format to skip marshalling / unmarshalling of the header record. Configure this parameter on the primary record (e.g., not the header or footer).

skipFooter

boolean

Camel 2.11 - optional - Configures the data format to skip marshalling / unmarshalling of the footer record Configure this parameter on the primary record (e.g., not the header or footer)..

isHeader

boolean

Camel 2.11 - optional - Identifies this FixedLengthRecord as a header record

isFooter

boolean

Camel 2.11 - optional - Identifies this FixedLengthRecords as a footer record

ignoreTrailingChars

boolean

Camel 2.11.1 - optional - Indicates that characters beyond the last mapped filed can be ignored when unmarshalling / parsing.

 

 

This annotation is associated to the root class of the model and must be declared one time.

The hasHeader/hasFooter parameters are mutually exclusive with isHeader/isFooter. A record may not be both a header/footer and a primary fixed-length record.

case 1 : Simple fixed length record

This simple example shows how to design the model to parse/format a fixed message

10A9PaulineMISINXD12345678BUYShare2500.45USD01-08-2009

Fixed-simple
   @FixedLengthRecord(length=54, paddingChar=' ')
    public static class Order {

        @DataField(pos = 1, length=2)
        private int orderNr;

        @DataField(pos = 3, length=2)
        private String clientNr;

        @DataField(pos = 5, length=7)
        private String firstName;

        @DataField(pos = 12, length=1, align="L")
        private String lastName;

        @DataField(pos = 13, length=4)
        private String instrumentCode;

        @DataField(pos = 17, length=10)
        private String instrumentNumber;

        @DataField(pos = 27, length=3)
        private String orderType;

        @DataField(pos = 30, length=5)
        private String instrumentType;

        @DataField(pos = 35, precision = 2, length=7)
        private BigDecimal amount;

        @DataField(pos = 42, length=3)
        private String currency;

        @DataField(pos = 45, length=10, pattern = "dd-MM-yyyy")
        private Date orderDate;
        ...

case 2 : Fixed length record with alignment and padding

This more elaborated example show how to define the alignment for a field and how to assign a padding character which is ' ' here''

10A9 PaulineM ISINXD12345678BUYShare2500.45USD01-08-2009

Fixed-padding-align
   @FixedLengthRecord(length=60, paddingChar=' ')
    public static class Order {

        @DataField(pos = 1, length=2)
        private int orderNr;

        @DataField(pos = 3, length=2)
        private String clientNr;

        @DataField(pos = 5, length=9)
        private String firstName;

        @DataField(pos = 14, length=5, align="L")   // align text to the LEFT zone of the block
        private String lastName;

        @DataField(pos = 19, length=4)
        private String instrumentCode;

        @DataField(pos = 23, length=10)
        private String instrumentNumber;

        @DataField(pos = 33, length=3)
        private String orderType;

        @DataField(pos = 36, length=5)
        private String instrumentType;

        @DataField(pos = 41, precision = 2, length=7)
        private BigDecimal amount;

        @DataField(pos = 48, length=3)
        private String currency;

        @DataField(pos = 51, length=10, pattern = "dd-MM-yyyy")
        private Date orderDate;
        ...

case 3 : Field padding

Sometimes, the default padding defined for record cannnot be applied to the field as we have a number format where we would like to padd with '0' instead of ' '. In this case, you can use in the model the attribute paddingField to set this value.

10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009

Fixed-padding-field
    @FixedLengthRecord(length = 65, paddingChar = ' ')
    public static class Order {

        @DataField(pos = 1, length = 2)
        private int orderNr;

        @DataField(pos = 3, length = 2)
        private String clientNr;

        @DataField(pos = 5, length = 9)
        private String firstName;

        @DataField(pos = 14, length = 5, align = "L")
        private String lastName;

        @DataField(pos = 19, length = 4)
        private String instrumentCode;

        @DataField(pos = 23, length = 10)
        private String instrumentNumber;

        @DataField(pos = 33, length = 3)
        private String orderType;

        @DataField(pos = 36, length = 5)
        private String instrumentType;

        @DataField(pos = 41, precision = 2, length = 12, paddingChar = '0')
        private BigDecimal amount;

        @DataField(pos = 53, length = 3)
        private String currency;

        @DataField(pos = 56, length = 10, pattern = "dd-MM-yyyy")
        private Date orderDate;
        ...

case 4: Fixed length record with delimiter

Fixed-length records sometimes have delimited content within the record. The firstName and lastName fields are delimited with the '^' character in the following example:

10A9Pauline^M^ISINXD12345678BUYShare000002500.45USD01-08-2009

Fixed-delimited
    @FixedLengthRecord()
    public static class Order {

        @DataField(pos = 1, length = 2)
        private int orderNr;

        @DataField(pos = 2, length = 2)
        private String clientNr;

        @DataField(pos = 3, delimiter = "^")
        private String firstName;

        @DataField(pos = 4, delimiter = "^")
        private String lastName;

        @DataField(pos = 5, length = 4)
        private String instrumentCode;

        @DataField(pos = 6, length = 10)
        private String instrumentNumber;

        @DataField(pos = 7, length = 3)
        private String orderType;

        @DataField(pos = 8, length = 5)
        private String instrumentType;

        @DataField(pos = 9, precision = 2, length = 12, paddingChar = '0')
        private BigDecimal amount;

        @DataField(pos = 10, length = 3)
        private String currency;

        @DataField(pos = 11, length = 10, pattern = "dd-MM-yyyy")
        private Date orderDate;

As of Camel 2.11 the 'pos' value(s) in a fixed-length record may optionally be defined using ordinal, sequential values instead of precise column numbers.

case 5 : Fixed length record with record-defined field length

Occasionally a fixed-length record may contain a field that define the expected length of another field within the same record. In the following example the length of the instrumentNumber field value is defined by the value of instrumentNumberLen field in the record.

10A9Pauline^M^ISIN10XD12345678BUYShare000002500.45USD01-08-2009

Fixed-delimited
    @FixedLengthRecord()
    public static class Order {

        @DataField(pos = 1, length = 2)
        private int orderNr;

        @DataField(pos = 2, length = 2)
        private String clientNr;

        @DataField(pos = 3, delimiter = "^")
        private String firstName;

        @DataField(pos = 4, delimiter = "^")
        private String lastName;

        @DataField(pos = 5, length = 4)
        private String instrumentCode;

        @DataField(pos = 6, length = 2, align = "R", paddingChar = '0')
        private int instrumentNumberLen;
        
        @DataField(pos = 7, lengthPos=6)
        private String instrumentNumber;

        @DataField(pos = 8, length = 3)
        private String orderType;

        @DataField(pos = 9, length = 5)
        private String instrumentType;

        @DataField(pos = 10, precision = 2, length = 12, paddingChar = '0')
        private BigDecimal amount;

        @DataField(pos = 11, length = 3)
        private String currency;

        @DataField(pos = 12, length = 10, pattern = "dd-MM-yyyy")
        private Date orderDate;

case 6 : Fixed length record with header and footer

Bindy will discover fixed-length header and footer records that are configured as part of the model – provided that the annotated classes exist either in the same package as the primary @FixedLengthRecord class, or within one of the configured scan packages. The following text illustrates two fixed-length records that are bracketed by a header record and footer record.

101-08-2009
10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009
10A9 RichN ISINXD12345678BUYShare000002700.45USD01-08-2009
9000000002

Fixed-header-and-footer-main-class
@FixedLengthRecord(hasHeader = true, hasFooter = true)
public class Order {

    @DataField(pos = 1, length = 2)
    private int orderNr;

    @DataField(pos = 2, length = 2)
    private String clientNr;

    @DataField(pos = 3, length = 9)
    private String firstName;

    @DataField(pos = 4, length = 5, align = "L")
    private String lastName;

    @DataField(pos = 5, length = 4)
    private String instrumentCode;

    @DataField(pos = 6, length = 10)
    private String instrumentNumber;

    @DataField(pos = 7, length = 3)
    private String orderType;

    @DataField(pos = 8, length = 5)
    private String instrumentType;

    @DataField(pos = 9, precision = 2, length = 12, paddingChar = '0')
    private BigDecimal amount;

    @DataField(pos = 10, length = 3)
    private String currency;

    @DataField(pos = 11, length = 10, pattern = "dd-MM-yyyy")
    private Date orderDate;
...
}


@FixedLengthRecord(isHeader = true)
public  class OrderHeader {
    @DataField(pos = 1, length = 1)
    private int recordType = 1;
    
    @DataField(pos = 2, length = 10, pattern = "dd-MM-yyyy")
    private Date recordDate;
    
...
}


@FixedLengthRecord(isFooter = true)
public class OrderFooter {
    
    @DataField(pos = 1, length = 1)
    private int recordType = 9;
    
    @DataField(pos = 2, length = 9, align = "R", paddingChar = '0')
    private int numberOfRecordsInTheFile;

...
}

case 7 : Skipping content when parsing a fixed length record. (Camel 2.11.1)

It is common to integrate with systems that provide fixed-length records containing more information than needed for the target use case. It is useful in this situation to skip the declaration and parsing of those fields that we do not need. To accomodate this, Bindy will skip forward to the next mapped field within a record if the 'pos' value of the next declared field is beyond the cursor position of the last parsed field. Using absolute 'pos' locations for the fields of interest (instead of ordinal values) causes Bindy to skip content between two fields.

Similarly, it is possible that none of the content beyond some field is of interest. In this case, you can tell Bindy to skip parsing of everything beyond the last mapped field by setting the ignoreTrailingChars property on the @FixedLengthRecord declaration.

@FixedLengthRecord(ignoreTrailingChars = true)
public static class Order {

        @DataField(pos = 1, length = 2)
        private int orderNr;

        @DataField(pos = 3, length = 2)
        private String clientNr;

    ... any characters that appear beyond the last mapped field will be ignored

}

5. Message

The Message annotation is used to identified the class of your model who will contain key value pairs fields. This kind of format is used mainly in Financial Exchange Protocol Messages (FIX). Nevertheless, this annotation can be used for any other format where data are identified by keys. The key pair values are separated each other by a separator which can be a special character like a tab delimitor (unicode representation : \u0009) or a start of heading (unicode representation : \u0001)

"FIX information"

More information about FIX can be found on this web site : http://www.fixprotocol.org/. To work with FIX messages, the model must contain a Header and Trailer classes linked to the root message class which could be a Order class. This is not mandatory but will be very helpful when you will use camel-bindy in combination with camel-fix which is a Fix gateway based on quickFix project http://www.quickfixj.org/.

Annotation name

Record type

Level

Message

key value pair

Class

Parameter name

type

Info

pairSeparator

string

mandatory - can be '=' or ';' or 'anything'

keyValuePairSeparair

string

mandatory - can be '\u0001', '\u0009', '#' or 'anything'

crlf

string

optional - possible values = WINDOWS,UNIX,MAC, or custom; default value = WINDOWS - allow to define the carriage return character to use. If you specify a value other than the three listed before, the value you enter (custom) will be used as the CRLF character(s)

type

string

optional - define the type of message (e.g. FIX, EMX, ...)

version

string

optional - version of the message (e.g. 4.1)

isOrdered

boolean

optional - default value = false - allow to change the order of the fields when FIX message is generated

 

 

This annotation is associated to the message class of the model and must be declared one time.

case 1 : separator = 'u0001'

The separator used to segregate the key value pair fields in a FIX message is the ASCII '01' character or in unicode format '\u0001'. This character must be escaped a second time to avoid a java runtime error. Here is an example :

8=FIX.4.1 9=20 34=1 35=0 49=INVMGR 56=BRKR 1=BE.CHM.001 11=CHM0001-01 22=4 ...

and how to use the annotation

FIX - message
@Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1")
public class Order {
...
}

Look at test cases

The ASCII character like tab, ... cannot be displayed in WIKI page. So, have a look to the test case of camel-bindy to see exactly how the FIX message looks like (src\test\data\fix\fix.txt) and the Order, Trailer, Header classes (src\test\java\org\apache\camel\dataformat\bindy\model\fix\simple\Order.java)

6. KeyValuePairField

The KeyValuePairField annotation defines the property of a key value pair field. Each KeyValuePairField is identified by a tag (= key) and its value associated, a type (string, int, date, ...), optionaly a pattern and if the field is required

Annotation name

Record type

Level

KeyValuePairField

Key Value Pair - FIX

Property

Parameter name

type

Info

tag

int

mandatory - digit number identifying the field in the message - must be unique

pattern

string

optional - default value = "" - will be used to format Decimal, Date, ...

precision

int

optional - digit number - represents the precision to be used when the Decimal number will be formatted/parsed

position

int

optional - must be used when the position of the key/tag in the FIX message must be different

required

boolean

optional - default value = "false"

impliedDecimalSeparator

boolean

Camel 2.11: optional - default value = "false" - Indicates if there is a decimal point implied at a specified location

case 1 : tag

This parameter represents the key of the field in the message

FIX message - Tag
@Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1")
public class Order {

    @Link Header header;

    @Link Trailer trailer;

    @KeyValuePairField(tag = 1) // Client reference
    private String Account;

    @KeyValuePairField(tag = 11) // Order reference
    private String ClOrdId;

    @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...)
    private String IDSource;

    @KeyValuePairField(tag = 48) // Fund code
    private String SecurityId;

    @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell)
    private String Side;

    @KeyValuePairField(tag = 58) // Free text
    private String Text;

...
}

case 2 : Different position in output

If the tags/keys that we will put in the FIX message must be sorted according to a predefine order, then use the attribute 'position' of the annotation @KeyValuePairField

FIX message - Tag - sort
@Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true)
public class Order {

    @Link Header header;

    @Link Trailer trailer;

    @KeyValuePairField(tag = 1, position = 1) // Client reference
    private String account;

    @KeyValuePairField(tag = 11, position = 3) // Order reference
    private String clOrdId;

...
}

7. Section

In FIX message of fixed length records, it is common to have different sections in the representation of the information : header, body and section. The purpose of the annotation @Section is to inform bindy about which class of the model represents the header (= section 1), body (= section 2) and footer (= section 3)

Only one attribute/parameter exists for this annotation.

Annotation name

Record type

Level

Section

FIX

Class

Parameter name

type

Info

number

int

digit number identifying the section position

case 1 : Section

A. Definition of the header section

FIX message - Section - Header
@Section(number = 1)
public class Header {

    @KeyValuePairField(tag = 8, position = 1) // Message Header
    private String beginString;

    @KeyValuePairField(tag = 9, position = 2) // Checksum
    private int bodyLength;
...
}

B. Definition of the body section

FIX message - Section - Body
@Section(number = 2)
@Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true)
public class Order {

    @Link Header header;

    @Link Trailer trailer;

    @KeyValuePairField(tag = 1, position = 1) // Client reference
    private String account;

    @KeyValuePairField(tag = 11, position = 3) // Order reference
    private String clOrdId;

C. Definition of the footer section

FIX message - Section - Footer
@Section(number = 3)
public class Trailer {

    @KeyValuePairField(tag = 10, position = 1)
    // CheckSum
    private int checkSum;

    public int getCheckSum() {
        return checkSum;
    }

8. OneToMany

The purpose of the annotation @OneToMany is to allow to work with a List<?> field defined a POJO class or from a record containing repetitive groups.

Restrictions OneToMany

Be careful, the one to many of bindy does not allow to handle repetitions defined on several levels of the hierarchy

The relation OneToMany ONLY WORKS in the following cases :

  • Reading a FIX message containing repetitive groups (= group of tags/keys)
  • Generating a CSV with repetitive data

Annotation name

Record type

Level

OneToMany

all

property

Parameter name

type

Info

mappedTo

string

optional - string - class name associated to the type of the List<Type of the Class>

case 1 : Generating CSV with repetitive data

Here is the CSV output that we want :

Claus,Ibsen,Camel in Action 1,2010,35
Claus,Ibsen,Camel in Action 2,2012,35
Claus,Ibsen,Camel in Action 3,2013,35
Claus,Ibsen,Camel in Action 4,2014,35

Remark : the repetitive data concern the title of the book and its publication date while first, last name and age are common

and the classes used to modeling this. The Author class contains a List of Book.

Generate CSV with repetitive data
@CsvRecord(separator=",")
public class Author {

	@DataField(pos = 1)
	private String firstName;

	@DataField(pos = 2)
	private String lastName;

	@OneToMany
	private List<Book> books;

	@DataField(pos = 5)
	private String Age;
...


public class Book {

	@DataField(pos = 3)
	private String title;

	@DataField(pos = 4)
	private String year;

Very simple isn't it !!!

case 2 : Reading FIX message containing group of tags/keys

Here is the message that we would like to process in our model :

"8=FIX 4.19=2034=135=049=INVMGR56=BRKR"
"1=BE.CHM.00111=CHM0001-0158=this is a camel - bindy test"
"22=448=BE000124567854=1"
"22=548=BE000987654354=2"
"22=648=BE000999999954=3"
"10=220"

tags 22, 48 and 54 are repeated

and the code

Reading FIX message containing group of tags/keys
public class Order {

    @Link Header header;

    @Link Trailer trailer;

    @KeyValuePairField(tag = 1) // Client reference
    private String account;

    @KeyValuePairField(tag = 11) // Order reference
    private String clOrdId;

    @KeyValuePairField(tag = 58) // Free text
    private String text;

    @OneToMany(mappedTo = "org.apache.camel.dataformat.bindy.model.fix.complex.onetomany.Security")
    List<Security> securities;
...

public class Security {

    @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...)
    private String idSource;

    @KeyValuePairField(tag = 48) // Fund code
    private String securityCode;

    @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell)
    private String side;

Using the Java DSL

The next step consists in instantiating the DataFormat bindy class associated with this record type and providing Java package name(s) as parameter.

For example the following uses the class BindyCsvDataFormat (who correspond to the class associated with the CSV record type) which is configured with "com.acme.model"
package name to initialize the model objects configured in this package.

// Camel 2.15 or older (configure by package name)
DataFormat bindy = new BindyCsvDataFormat("com.acme.model");

 
// Camel 2.16 onwards (configure by class name)
DataFormat bindy = new BindyCsvDataFormat(com.acme.model.MyModel.class);

Setting locale

Bindy supports configuring the locale on the dataformat, such as 

// Camel 2.15 or older (configure by package name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat("com.acme.model");
// Camel 2.16 onwards (configure by class name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat(com.acme.model.MyModel.class);

bindy.setLocale("us");

Or to use the platform default locale then use "default" as the locale name. Notice this requires Camel 2.14/2.13.3/2.12.5.

// Camel 2.15 or older (configure by package name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat("com.acme.model");
// Camel 2.16 onwards (configure by class name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat(com.acme.model.MyModel.class);

bindy.setLocale("default");

for older releases you can set it using Java code as shown

// Camel 2.15 or older (configure by package name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat("com.acme.model");
// Camel 2.16 onwards (configure by class name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat(com.acme.model.MyModel.class);


bindy.setLocale(Locale.getDefault().getISO3Country());

Unmarshaling

from("file://inbox")
  .unmarshal(bindy)
  .to("direct:handleOrders");

Alternatively, you can use a named reference to a data format which can then be defined in your Registry e.g. your Spring XML file:

from("file://inbox")
  .unmarshal("myBindyDataFormat")
  .to("direct:handleOrders");

The Camel route will pick-up files in the inbox directory, unmarshall CSV records into a collection of model objects and send the collection
to the route referenced by 'handleOrders'.

The collection returned is a List of Map objects. Each Map within the list contains the model objects that were marshalled out of each line of the CSV. The reason behind this is that each line can correspond to more than one object. This can be confusing when you simply expect one object to be returned per line.

Each object can be retrieve using its class name.

    List<Map<String, Object>> unmarshaledModels = (List<Map<String, Object>>) exchange.getIn().getBody();

    int modelCount = 0;
    for (Map<String, Object> model : unmarshaledModels) {
	  for (String className : model.keySet()) {
	     Object obj = model.get(className);
	     LOG.info("Count : " + modelCount + ", " + obj.toString());
	  }
	 modelCount++;
    }

    LOG.info("Total CSV records received by the csv bean : " + modelCount);

Assuming that you want to extract a single Order object from this map for processing in a route, you could use a combination of a Splitter and a Processor as per the following:

from("file://inbox")
    .unmarshal(bindy)
    .split(body())
        .process(new Processor() {
            public void process(Exchange exchange) throws Exception {
                Message in = exchange.getIn();
                Map<String, Object> modelMap = (Map<String, Object>) in.getBody();
                in.setBody(modelMap.get(Order.class.getCanonicalName()));
            }
        })
        .to("direct:handleSingleOrder")
    .end();

Take care of the fact that Bindy uses CHARSET_NAME property or the CHARSET_NAME header as define in the Exchange interface to do a characterset conversion of the inputstream received for unmarshalling. In some producers (e.g. file-endpoint) you can define a characterset. The characterset conversion can already been done by this producer. Sometimes you need to remove this property or header from the exchange before sending it to the unmarshal. If you don't remove it the conversion might be done twice which might lead to unwanted results.

from("file://inbox?charset=Cp922")
  .removeProperty(Exchange.CHARSET_NAME)
  .unmarshal("myBindyDataFormat")
  .to("direct:handleOrders");

Marshaling

To generate CSV records from a collection of model objects, you create the following route :

from("direct:handleOrders")
   .marshal(bindy)
   .to("file://outbox")

Using Spring XML

This is really easy to use Spring as your favorite DSL language to declare the routes to be used for camel-bindy. The following example shows two routes where the first will pick-up records from files, unmarshal the content and bind it to their model. The result is then send to a pojo (doing nothing special) and place them into a queue.

The second route will extract the pojos from the queue and marshal the content to generate a file containing the csv record. The example above is for using Camel 2.16 onwards.

spring dsl
<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd
       http://camel.apache.org/schema/spring
       http://camel.apache.org/schema/spring/camel-spring.xsd">

        <!-- Queuing engine - ActiveMq - work locally in mode virtual memory -->
	<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
		<property name="brokerURL" value="vm://localhost:61616"/>
	</bean>


	<camelContext xmlns="http://camel.apache.org/schema/spring">

 
        <dataFormats>
          <bindy id="bindyDataformat" type="Csv" classType="org.apache.camel.bindy.model.Order"/>
        </dataFormats>

		<route>
			<from uri="file://src/data/csv/?noop=true" />
			<unmarshal ref="bindyDataformat" />
			<to uri="bean:csv" />
			<to uri="activemq:queue:in" />
		</route>

		<route>
			<from uri="activemq:queue:in" />
			<marshal ref="bindyDataformat" />
			<to uri="file://src/data/csv/out/" />
		</route>
	</camelContext>
</beans>

Be careful

Please verify that your model classes implements serializable otherwise the queue manager will raise an error

Dependencies

To use Bindy in your camel routes you need to add the a dependency on camel-bindy which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-bindy</artifactId>
  <version>x.x.x</version>
</dependency>

XMLSecurity Data Format

The XMLSecurity Data Format facilitates encryption and decryption of XML payloads at the Document, Element, and Element Content levels (including simultaneous multi-node encryption/decryption using XPath). To sign messages using the XML Signature specification, please see the Camel XML Security component.

The encryption capability is based on formats supported using the Apache XML Security (Santuario) project. Symmetric encryption/decryption is currently supported using Triple-DES and AES (128, 192, and 256) encryption formats. Additional formats can be easily added later as needed. This capability allows Camel users to encrypt/decrypt payloads while being dispatched or received along a route.

Available as of Camel 2.9
The XMLSecurity Data Format supports asymmetric key encryption. In this encryption model a symmetric key is generated and used to perform XML content encryption or decryption. This "content encryption key" is then itself encrypted using an asymmetric encryption algorithm that leverages the recipient's public key as the "key encryption key". Use of an asymmetric key encryption algorithm ensures that only the holder of the recipient's private key can access the generated symmetric encryption key. Thus, only the private key holder can decode the message. The XMLSecurity Data Format handles all of the logic required to encrypt and decrypt the message content and encryption key(s) using asymmetric key encryption.

The XMLSecurity Data Format also has improved support for namespaces when processing the XPath queries that select content for encryption. A namespace definition mapping can be included as part of the data format configuration. This enables true namespace matching, even if the prefix values in the XPath query and the target xml document are not equivalent strings.

Basic Options

Option

Default

Description

secureTag

null

The XPath reference to the XML Element selected for encryption/decryption. If no tag is specified, the entire payload is encrypted/decrypted.

secureTagContents

false

A boolean value to specify whether the XML Element is to be encrypted or the contents of the XML Element

  • false = Element Level
  • true = Element Content Level

passPhrase

null

A String used as passPhrase to encrypt/decrypt content. The passPhrase has to be provided. If no passPhrase is specified, a default passPhrase is used. The passPhrase needs to be put together in conjunction with the appropriate encryption algorithm. For example using TRIPLEDES the passPhase can be a "Only another 24 Byte key"

xmlCipherAlgorithm

TRIPLEDES

The cipher algorithm to be used for encryption/decryption of the XML message content. The available choices are:

  • XMLCipher.TRIPLEDES
  • XMLCipher.AES_128
  • XMLCipher.AES_128_GCM Camel 2.12
  • XMLCipher.AES_192
  • XMLCipher.AES_192_GCM Camel 2.12
  • XMLCipher.AES_256
  • XMLCipher.AES_256_GCM Camel 2.12
  • XMLCipher.SEED_128 Camel 2.15
  • XMLCipher.CAMELLIA_128, XMLCipher.CAMELLIA_192, XMLCipher.CAMELLIA_256 Camel 2.15

namespaces

null

A map of namespace values indexed by prefix. The index values must match the prefixes used in the secureTag XPath query.

Asymmetric Encryption Options

These options can be applied in addition to relevant the Basic options to use asymmetric key encryption.

Option

Default

Description

recipientKeyAlias

null

The key alias to be used when retrieving the recipient's public or private key from a KeyStore when performing asymmetric key encryption or decryption.

keyCipherAlgorithm

Camel 2.12 XMLCipher.RSA_OAEP

The cipher algorithm to be used for encryption/decryption of the asymmetric key. The available choices are:

  • XMLCipher.RSA_v1dot5
  • XMLCipher.RSA_OAEP
  • XMLCipher.RSA_OAEP_11

keyOrTrustStoreParameters

null

Configuration options for creating and loading a KeyStore instance that represents the sender's trustStore or recipient's keyStore.

keyPassword

null

Camel 2.10.2 / 2.11: The password to be used for retrieving the private key from the KeyStore. This key is used for asymmetric decryption.

digestAlgorithm

XMLCipher.SHA1

Camel 2.12 The digest algorithm to use with the RSA OAEP algorithm. The available choices are:

  • XMLCipher.SHA1
  • XMLCipher.SHA256
  • XMLCipher.SHA512

mgfAlgorithm

EncryptionConstants.MGF1_SHA1

Camel 2.12 The MGF Algorithm to use with the RSA OAEP algorithm. The available choices are:

  • EncryptionConstants.MGF1_SHA1
  • EncryptionConstants.MGF1_SHA256
  • EncryptionConstants.MGF1_SHA512
addKeyValueForEncryptedKeytrueCamel 2.14.1 Whether to add the public key used to encrypt the session key as a KeyValue in the EncryptedKey structure or not.

Key Cipher Algorithm

As of Camel 2.12.0, the default Key Cipher Algorithm is now XMLCipher.RSA_OAEP instead of XMLCipher.RSA_v1dot5. Usage of XMLCipher.RSA_v1dot5 is discouraged due to various attacks. Requests that use RSA v1.5 as the key cipher algorithm will be rejected unless it has been explicitly configured as the key cipher algorithm.

Marshal

In order to encrypt the payload, the marshal processor needs to be applied on the route followed by the secureXML() tag.

Unmarshal

In order to decrypt the payload, the unmarshal processor needs to be applied on the route followed by the secureXML() tag.

Examples

Given below are several examples of how marshalling could be performed at the Document, Element, and Content levels.

Full Payload encryption/decryption

from("direct:start")
    .marshal().secureXML()
    .unmarshal().secureXML()
    .to("direct:end");

Partial Payload Content Only encryption/decryption

String tagXPATH = "//cheesesites/italy/cheese";
boolean secureTagContent = true;
...
from("direct:start")
    .marshal().secureXML(tagXPATH, secureTagContent)
    .unmarshal().secureXML(tagXPATH, secureTagContent)
    .to("direct:end");

Partial Multi Node Payload Content Only encryption/decryption

String tagXPATH = "//cheesesites/*/cheese";
boolean secureTagContent = true;
...
from("direct:start")
    .marshal().secureXML(tagXPATH, secureTagContent)
    .unmarshal().secureXML(tagXPATH, secureTagContent)
    .to("direct:end");

Partial Payload Content Only encryption/decryption with choice of passPhrase(password)

String tagXPATH = "//cheesesites/italy/cheese";
boolean secureTagContent = true;
...
String passPhrase = "Just another 24 Byte key";
from("direct:start")
    .marshal().secureXML(tagXPATH, secureTagContent, passPhrase)
    .unmarshal().secureXML(tagXPATH, secureTagContent, passPhrase)
    .to("direct:end");

Partial Payload Content Only encryption/decryption with passPhrase(password) and Algorithm

import org.apache.xml.security.encryption.XMLCipher;
....
String tagXPATH = "//cheesesites/italy/cheese";
boolean secureTagContent = true;
String passPhrase = "Just another 24 Byte key";
String algorithm= XMLCipher.TRIPLEDES;
from("direct:start")
    .marshal().secureXML(tagXPATH, secureTagContent, passPhrase, algorithm)
    .unmarshal().secureXML(tagXPATH, secureTagContent, passPhrase, algorithm)
    .to("direct:end");

Partial Payload Content with Namespace support

Java DSL
final Map<String, String> namespaces = new HashMap<String, String>();
namespaces.put("cust", "http://cheese.xmlsecurity.camel.apache.org/");

final KeyStoreParameters tsParameters = new KeyStoreParameters();
tsParameters.setPassword("password");
tsParameters.setResource("sender.ts");

context.addRoutes(new RouteBuilder() {
    public void configure() {
        from("direct:start")
           .marshal().secureXML("//cust:cheesesites/italy", namespaces, true, "recipient",
                                testCypherAlgorithm, XMLCipher.RSA_v1dot5, tsParameters)
           .to("mock:encrypted");
    }
}
Spring XML

A namespace prefix that is defined as part of the camelContext definition can be re-used in context within the data format secureTag attribute of the secureXML element.

<camelContext id="springXmlSecurityDataFormatTestCamelContext" 
              xmlns="http://camel.apache.org/schema/spring"
              xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/">        
    <route>
        <from uri="direct://start"/>
            <marshal>
                <secureXML secureTag="//cheese:cheesesites/italy"
                           secureTagContents="true"/>
            </marshal> 
            ...

Asymmetric Key Encryption

Spring XML Sender
<!--  trust store configuration -->                          
<camel:keyStoreParameters id="trustStoreParams" resource="./sender.ts" password="password"/>

<camelContext id="springXmlSecurityDataFormatTestCamelContext" 
              xmlns="http://camel.apache.org/schema/spring"
              xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/">        
    <route>
        <from uri="direct://start"/>
            <marshal>
                <secureXML secureTag="//cheese:cheesesites/italy"
                           secureTagContents="true"
                           xmlCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc"       
                           keyCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"
                           recipientKeyAlias="recipient"
                           keyOrTrustStoreParametersId="trustStoreParams"/>
            </marshal> 
            ...
Spring XML Recipient
 
<!--  key store configuration -->
<camel:keyStoreParameters id="keyStoreParams" resource="./recipient.ks" password="password" />

<camelContext id="springXmlSecurityDataFormatTestCamelContext" 
              xmlns="http://camel.apache.org/schema/spring"
              xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/">
    <route>    
        <from uri="direct://encrypted"/>
            <unmarshal>
                <secureXML secureTag="//cheese:cheesesites/italy"
                           secureTagContents="true"
                           xmlCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc"
                           keyCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5"
                           recipientKeyAlias="recipient"
                           keyOrTrustStoreParametersId="keyStoreParams"
                           keyPassword="privateKeyPassword" />
            </unmarshal>
            ...

Dependencies

This data format is provided within the camel-xmlsecurity component.

The GZip Data Format is a message compression and de-compression format. It uses the same deflate algorithm that is used in Zip DataFormat, although some additional headers are provided. This format is produced by popular gzip/gunzip tool. Messages marshalled using GZip compression can be unmarshalled using GZip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads or when you read messages previously comressed using gzip tool.

Options

There are no options provided for this data format.

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing gzip compression format and send it an ActiveMQ queue called MY_QUEUE.

from("direct:start").marshal().gzip().to("activemq:queue:MY_QUEUE");

Unmarshal

In this example we unmarshal a gzipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnGZippedMessageProcessor.

from("activemq:queue:MY_QUEUE").unmarshal().gzip().process(new UnGZippedMessageProcessor()); 

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

Castor

Available as of Camel 2.1

Castor is a Data Format which uses the Castor XML library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

As usually you can use either Java DSL or Spring XML to work with Castor Data Format.

Using the Java DSL

from("direct:order").
  marshal().castor().
  to("activemq:queue:order");

For example the following uses a named DataFormat of Castor which uses default Castor data binding features.

CastorDataFormat castor = new CastorDataFormat ();

from("activemq:My.Queue").
  unmarshal(castor).
  to("mqseries:Another.Queue");

If you prefer to use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g.

from("activemq:My.Queue").
  unmarshal("mycastorType").
  to("mqseries:Another.Queue");

If you want to override default mapping schema by providing a mapping file you can set it as follows.

CastorDataFormat castor = new CastorDataFormat ();
castor.setMappingFile("mapping.xml");

Also if you want to have more control on Castor Marshaller and Unmarshaller you can access them as below.

castor.getMarshaller();
castor.getUnmarshaller();

Using Spring XML

The following example shows how to use Castor to unmarshal using Spring configuring the castor data type

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
  <route>
    <from uri="direct:start"/>
    <unmarshal>
      <castor validation="true" />
    </unmarshal>
    <to uri="mock:result"/>
  </route>
</camelContext>

This example shows how to configure the data type just once and reuse it on multiple routes. You have to set the <castor> element directly in <camelContext>.

<camelContext>
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
  <dataFormats>
    <castor id="myCastor"/>
  </dataFormats>

  <route>
    <from uri="direct:start"/>
    <marshal ref="myCastor"/>
    <to uri="direct:marshalled"/>
  </route>
  <route>
    <from uri="direct:marshalled"/>
    <unmarshal ref="myCastor"/>
    <to uri="mock:result"/>
  </route>

</camelContext>

Options

Castor supports the following options

Option

Type

Default

Description

encoding

String

UTF-8

Encoding to use when marshalling an Object to XML

validation

Boolean

false

Whether validation is turned on or off.

mappingFile

String

null

Path to a Castor mapping file to load from the classpath.

packages

String[]

null

Add additional packages to Castor XmlContext

classNames

String[]

null

Add additional class names to Castor XmlContext

Dependencies

To use Castor in your camel routes you need to add the a dependency on camel-castor which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-castor</artifactId>
  <version>x.x.x</version>
</dependency>

Protobuf - Protocol Buffers

"Protocol Buffers - Google's data interchange format"

Available from Camel 2.2

Camel provides a Data Format to serialse between Java and the Protocol Buffer protocol. The project's site details why you may wish to choose this format over xml. Protocol Buffer is language-neutral and platform-neutral, so messages produced by your Camel routes may be consumed by other language implementations.

API Site
Protobuf Implementation
Protobuf Java Tutorial

Protobuf overview

This quick overview of how to use Protobuf. For more detail see the complete tutorial

Defining the proto format

The first step is to define the format for the body of your exchange. This is defined in a .proto file as so:

addressbook.proto

package org.apache.camel.component.protobuf;

option java_package = "org.apache.camel.component.protobuf";
option java_outer_classname = "AddressBookProtos";

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}

Generating Java classes

The Protobuf SDK provides a compiler which will generate the Java classes for the format we defined in our .proto file. You can run the compiler for any additional supported languages you require.

protoc --java_out=. ./addressbook.proto

This will generate a single Java class named AddressBookProtos which contains inner classes for Person and AddressBook. Builders are also implemented for you. The generated classes implement com.google.protobuf.Message which is required by the serialisation mechanism. For this reason it important that only these classes are used in the body of your exchanges. Camel will throw an exception on route creation if you attempt to tell the Data Format to use a class that does not implement com.google.protobuf.Message. Use the generated builders to translate the data from any of your existing domain classes.

Java DSL

You can use create the ProtobufDataFormat instance and pass it to Camel DataFormat marshal and unmarsha API like this.

   ProtobufDataFormat format = new ProtobufDataFormat(Person.getDefaultInstance());

   from("direct:in").marshal(format);
   from("direct:back").unmarshal(format).to("mock:reverse");

Or use the DSL protobuf() passing the unmarshal default instance or default instance class name like this.

   // You don't need to specify the default instance for protobuf marshaling               
   from("direct:marshal").marshal().protobuf();
   from("direct:unmarshalA").unmarshal().
       protobuf("org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person").
       to ("mock:reverse");
                
   from("direct:unmarshalB").unmarshal().protobuf(Person.getDefaultInstance()).to("mock:reverse");

Spring DSL

The following example shows how to use Castor to unmarshal using Spring configuring the protobuf data type

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
  <route>
    <from uri="direct:start"/>
    <unmarshal>
      <protobuf instanceClass="org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person" />
    </unmarshal>
    <to uri="mock:result"/>
  </route>
</camelContext>

Dependencies

To use Protobuf in your camel routes you need to add the a dependency on camel-protobuf which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-protobuf</artifactId>
  <version>2.2.0</version>
</dependency>

SOAP DataFormat

Available as of Camel 2.3

SOAP is a Data Format which uses JAXB2 and JAX-WS annotations to marshal and unmarshal SOAP payloads. It provides the basic features of Apache CXF without need for the CXF Stack.

Supported SOAP versions

SOAP 1.1 is supported by default. SOAP 1.2 is supported from Camel 2.11 onwards.

Namespace prefix mapping

See JAXB for details how you can control namespace prefix mappings when marshalling using SOAP data format.

ElementNameStrategy

An element name strategy is used for two purposes. The first is to find a xml element name for a given object and soap action when marshaling the object into a SOAP message. The second is to find an Exception class for a given soap fault name.

Strategy

Usage

QNameStrategy

Uses a fixed qName that is configured on instantiation. Exception lookup is not supported

TypeNameStrategy

Uses the name and namespace from the @XMLType annotation of the given type. If no namespace is set then package-info is used. Exception lookup is not supported

ServiceInterfaceStrategy

Uses information from a webservice interface to determine the type name and to find the exception class for a SOAP fault

If you have generated the web service stub code with cxf-codegen or a similar tool then you probably will want to use the ServiceInterfaceStrategy. In the case you have no annotated service interface you should use QNameStrategy or TypeNameStrategy.

Using the Java DSL

The following example uses a named DataFormat of soap which is configured with the package com.example.customerservice to initialize the JAXBContext. The second parameter is the ElementNameStrategy. The route is able to marshal normal objects as well as exceptions. (Note the below just sends a SOAP Envelope to a queue. A web service provider would actually need to be listening to the queue for a SOAP call to actually occur, in which case it would be a one way SOAP request. If you need request reply then you should look at the next example.)

SoapJaxbDataFormat soap = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class));
from("direct:start")
  .marshal(soap)
  .to("jms:myQueue");

See also

As the SOAP dataformat inherits from the JAXB dataformat most settings apply here as well

Using SOAP 1.2

Available as of Camel 2.11

SoapJaxbDataFormat soap = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class));
soap.setVersion("1.2");
from("direct:start")
  .marshal(soap)
  .to("jms:myQueue");

When using XML DSL there is a version attribute you can set on the <soapjaxb> element.

    <!-- Defining a ServiceInterfaceStrategy for retrieving the element name when marshalling -->
    <bean id="myNameStrategy" class="org.apache.camel.dataformat.soap.name.ServiceInterfaceStrategy">
    	<constructor-arg value="com.example.customerservice.CustomerService"/>
	<constructor-arg value="true"/>
    </bean>

And in the Camel route

<route>
  <from uri="direct:start"/>
  <marshal>
    <soapjaxb contentPath="com.example.customerservice" version="1.2" elementNameStrategyRef="myNameStrategy"/>
  </marshal>
  <to uri="jms:myQueue"/>
</route>

Multi-part Messages

Available as of Camel 2.8.1

Multi-part SOAP messages are supported by the ServiceInterfaceStrategy. The ServiceInterfaceStrategy must be initialized with a service interface definition that is annotated in accordance with JAX-WS 2.2 and meets the requirements of the Document Bare style. The target method must meet the following criteria, as per the JAX-WS specification: 1) it must have at most one in or in/out non-header parameter, 2) if it has a return type other than void it must have no in/out or out non-header parameters, 3) if it it has a return type of void it must have at most one in/out or out non-header parameter.

The ServiceInterfaceStrategy should be initialized with a boolean parameter that indicates whether the mapping strategy applies to the request parameters or response parameters.

ServiceInterfaceStrategy strat =  new ServiceInterfaceStrategy(com.example.customerservice.multipart.MultiPartCustomerService.class, true);
SoapJaxbDataFormat soapDataFormat = new SoapJaxbDataFormat("com.example.customerservice.multipart", strat);

Multi-part Request

The payload parameters for a multi-part request are initiazlied using a BeanInvocation object that reflects the signature of the target operation. The camel-soap DataFormat maps the content in the BeanInvocation to fields in the SOAP header and body in accordance with the JAX-WS mapping when the marshal() processor is invoked.

BeanInvocation beanInvocation = new BeanInvocation();

// Identify the target method
beanInvocation.setMethod(MultiPartCustomerService.class.getMethod("getCustomersByName", 
    GetCustomersByName.class, com.example.customerservice.multipart.Product.class));

// Populate the method arguments
GetCustomersByName getCustomersByName = new GetCustomersByName();
getCustomersByName.setName("Dr. Multipart");
                
Product product = new Product();
product.setName("Multiuse Product");
product.setDescription("Useful for lots of things.");
                
Object[] args = new Object[] {getCustomersByName, product};

// Add the arguments to the bean invocation
beanInvocation.setArgs(args);

// Set the bean invocation object as the message body
exchange.getIn().setBody(beanInvocation); 

Multi-part Response

A multi-part soap response may include an element in the soap body and will have one or more elements in the soap header. The camel-soap DataFormat will unmarshall the element in the soap body (if it exists) and place it onto the body of the out message in the exchange. Header elements will not be marshaled into their JAXB mapped object types. Instead, these elements are placed into the camel out message header org.apache.camel.dataformat.soap.UNMARSHALLED_HEADER_LIST. The elements will appear either as element instance values, or as JAXBElement values, depending upon the setting for the ignoreJAXBElement property. This property is inherited from camel-jaxb.

You can also have the camel-soap DataFormate ignore header content all-together by setting the ignoreUnmarshalledHeaders value to true.

Holder Object mapping

JAX-WS specifies the use of a type-parameterized javax.xml.ws.Holder object for In/Out and Out parameters. A Holder object may be used when building the BeanInvocation, or you may use an instance of the parameterized-type directly. The camel-soap DataFormat marshals Holder values in accordance with the JAXB mapping for the class of the Holder's value. No mapping is provided for Holder objects in an unmarshalled response.

Examples

Webservice client

The following route supports marshalling the request and unmarshalling a response or fault.

String WS_URI = "cxf://http://myserver/customerservice?serviceClass=com.example.customerservice&dataFormat=MESSAGE";
SoapJaxbDataFormat soapDF = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class));
from("direct:customerServiceClient")
  .onException(Exception.class)
    .handled(true)
    .unmarshal(soapDF)
  .end()
  .marshal(soapDF)
  .to(WS_URI)
  .unmarshal(soapDF);

The below snippet creates a proxy for the service interface and makes a SOAP call to the above route.

import org.apache.camel.Endpoint;
import org.apache.camel.component.bean.ProxyHelper;
...

Endpoint startEndpoint = context.getEndpoint("direct:customerServiceClient");
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
// CustomerService below is the service endpoint interface, *not* the javax.xml.ws.Service subclass
CustomerService proxy = ProxyHelper.createProxy(startEndpoint, classLoader, CustomerService.class);
GetCustomersByNameResponse response = proxy.getCustomersByName(new GetCustomersByName());

Webservice Server

Using the following route sets up a webservice server that listens on jms queue customerServiceQueue and processes requests using the class CustomerServiceImpl. The customerServiceImpl of course should implement the interface CustomerService. Instead of directly instantiating the server class it could be defined in a spring context as a regular bean.

SoapJaxbDataFormat soapDF = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class));
CustomerService serverBean = new CustomerServiceImpl();
from("jms://queue:customerServiceQueue")
  .onException(Exception.class)
    .handled(true)
    .marshal(soapDF)
  .end()
  .unmarshal(soapDF)
  .bean(serverBean)
  .marshal(soapDF);

Dependencies

To use the SOAP dataformat in your camel routes you need to add the following dependency to your pom.

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-soap</artifactId>
  <version>2.3.0</version>
</dependency>

Crypto

Available as of Camel 2.3
PGP Available as of Camel 2.9

The Crypto Data Format integrates the Java Cryptographic Extension into Camel, allowing simple and flexible encryption and decryption of messages using Camel's familiar marshall and unmarshal formatting mechanism. It assumes marshalling to mean encryption to cyphertext and unmarshalling to mean decryption back to the original plaintext. This data format implements only symmetric (shared-key) encryption and decyption.

Options

Name

Type

Default

Description

algorithm

String

DES/CBC/PKCS5Padding

The JCE algorithm name indicating the cryptographic algorithm that will be used.

algorithmParameterSpec

java.security.spec.AlgorithmParameterSpec

null

A JCE AlgorithmParameterSpec used to initialize the Cipher.

bufferSize

Integer

4096

the size of the buffer used in the signature process.

cryptoProvider

String

null

The name of the JCE Security Provider that should be used.

initializationVector

byte[]

null

A byte array containing the Initialization Vector that will be used to initialize the Cipher.

inline

boolean

false

Flag indicating that the configured IV should be inlined into the encrypted data stream.

macAlgorithm

String

null

The JCE algorithm name indicating the Message Authentication algorithm.

shouldAppendHMAC

boolean

null

Flag indicating that a Message Authentication Code should be calculated and appended to the encrypted data.

Basic Usage

At its most basic all that is required to encrypt/decrypt an exchange is a shared secret key. If one or more instances of the Crypto data format are configured with this key the format can be used to encrypt the payload in one route (or part of one) and decrypted in another. For example, using the Java DSL as follows:{snippet:id=basic|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}In Spring the dataformat is configured first and then used in routes

xml<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <crypto id="basic" algorithm="DES" keyRef="desKey" /> </dataFormats> ... <route> <from uri="direct:basic-encryption" /> <marshal ref="basic" /> <to uri="mock:encrypted" /> <unmarshal ref="basic" /> <to uri="mock:unencrypted" /> </route> </camelContext>

Specifying the Encryption Algorithm

Changing the algorithm is a matter of supplying the JCE algorithm name. If you change the algorithm you will need to use a compatible key.{snippet:id=algorithm|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}A list of the available algorithms in Java 7 is available via the Java Cryptography Architecture Standard Algorithm Name Documentation.

Specifying an Initialization Vector

Some crypto algorithms, particularly block algorithms, require configuration with an initial block of data known as an Initialization Vector. In the JCE this is passed as an AlgorithmParameterSpec when the Cipher is initialized. To use such a vector with the CryptoDataFormat you can configure it with a byte[] containing the required data e.g.{snippet:id=init-vector|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}or with spring, suppling a reference to a byte[]{snippet:id=init-vector|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringCryptoDataFormatTest.xml}The same vector is required in both the encryption and decryption phases. As it is not necessary to keep the IV a secret, the DataFormat allows for it to be inlined into the encrypted data and subsequently read out in the decryption phase to initialize the Cipher. To inline the IV set the /oinline flag.{snippet:id=inline-init-vector|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}or with spring.{snippet:id=inline|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringCryptoDataFormatTest.xml}For more information of the use of Initialization Vectors, consult

Hashed Message Authentication Codes (HMAC)

To avoid attacks against the encrypted data while it is in transit the CryptoDataFormat can also calculate a Message Authentication Code for the encrypted exchange contents based on a configurable MAC algorithm. The calculated HMAC is appended to the stream after encryption. It is separated from the stream in the decryption phase. The MAC is recalculated and verified against the transmitted version to insure nothing was tampered with in transit.For more information on Message Authentication Codes see http://en.wikipedia.org/wiki/HMAC{snippet:id=hmac|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}or with spring.{snippet:id=hmac|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringCryptoDataFormatTest.xml}By default the HMAC is calculated using the HmacSHA1 mac algorithm though this can be easily changed by supplying a different algorithm name. See here for how to check what algorithms are available through the configured security providers{snippet:id=hmac-algorithm|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}or with spring.{snippet:id=hmac-algorithm|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringCryptoDataFormatTest.xml}

Supplying Keys Dynamically

When using a Recipient list or similar EIP the recipient of an exchange can vary dynamically. Using the same key across all recipients may neither be feasible or desirable. It would be useful to be able to specify keys dynamically on a per exchange basis. The exchange could then be dynamically enriched with the key of its target recipient before being processed by the data format. To facilitate this the DataFormat allow for keys to be supplied dynamically via the message headers below

  • CryptoDataFormat.KEY "CamelCryptoKey"

{snippet:id=key-in-header|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/CryptoDataFormatTest.java}or with spring.{snippet:id=header-key|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringCryptoDataFormatTest.xml}

PGP Message

The PGP Data Formater can create and decrypt/verify PGP Messages of the following PGP packet structure (entries in brackets are optional and ellipses indicate repetition, comma represents  sequential composition, and vertical bar separates alternatives):

    Public Key Encrypted Session Key ..., Symmetrically Encrypted Data | Sym. Encrypted and Integrity Protected Data, (Compressed Data,) (One Pass Signature ...,) Literal Data, (Signature ...,)

Since Camel 2.16.0 the Compressed Data packet is optional, before it was mandatory.

 

PGPDataFormat Options

Name

Type

Default

Description

keyUserid

String

null

The user ID of the key in the PGP keyring used during encryption. See also option keyUserids. Can also be only a part of a user ID. For example, if the user ID is "Test User <test@camel.com>" then you can use the part "Test User" or "<test@camel.com>" to address the user ID.

keyUserids

List<String>

null

Since camel 2.12.2: PGP allows to encrypt the symmetric key by several asymmetric public receiver keys. You can specify here the User IDs or parts of User IDs of several public keys contained in the PGP keyring. If you just have one User ID, then you can also use the option keyUserid. The User ID specified in keyUserid and the User IDs in keyUserids will be merged together and the corresponding public keys will be used for the encryption.

password

String

null

Password used when opening the private key (not used for encryption).

keyFileName

String

null

Filename of the keyring; must be accessible as a classpath resource (but you can specify a location in the file system by using the "file:" prefix).

encryptionKeyRing

byte[]

null

Since camel 2.12.1: encryption keyring; you can not set the keyFileName and encryptionKeyRing at the same time.

signatureKeyUserid

String

null

Since Camel 2.11.0; optional User ID of the key in the PGP keyring used for signing (during encryption) or signature verification (during decryption). During the signature verification process the specified User ID restricts the public keys from the public keyring which can be used for the verification. If no User ID is specified for the signature verficiation then any public key in the public keyring can be used for the verification. Can also be only a part of a user ID. For example, if the user ID is "Test User <test@camel.com>" then you can use the part "Test User" or "<test@camel.com>" to address the User ID.

signatureKeyUserids

List<String>

null

Since Camel 2.12.3: optional list of User IDs of the key in the PGP keyring used for signing (during encryption) or signature verification (during decryption). You can specify here the User IDs or parts of User IDs of several keys contained in the PGP keyring. If you just have one User ID, then you can also use the option keyUserid. The User ID specified in keyUserid and the User IDs in keyUserids will be merged together and the corresponding keys will be used for the signing or signature verification. If the specified User IDs reference several keys then for each key a signature is added to the PGP result during the encryption-signing process. In the decryption-verifying process the list of User IDs restricts the list of public keys which can be used for signature verification. If the list of User IDs is empty then any public key in the public keyring can be used for the signature verification.

signaturePassword

String

null

Since Camel 2.11.0: optional password used when opening the private key used for signing (during encryption).

signatureKeyFileName

String

null

Since Camel 2.11.0: optional filename of the keyring to use for signing (during encryption) or for signature verification (during decryption); must be accessible as a classpath resource (but you can specify a location in the file system by using the "file:" prefix).

signatureKeyRing

byte[]

null

Since camel 2.12.1: signature keyring; you can not set the signatureKeyFileName and signatureKeyRing at the same time.

algorithm

int

SymmetricKeyAlgorithmTags.CAST5

Since camel 2.12.2: symmetric key encryption algorithm; possible values are defined in org.bouncycastle.bcpg.SymmetricKeyAlgorithmTags; for example 2 (= TRIPLE DES), 3 (= CAST5), 4 (= BLOWFISH), 6 (= DES), 7 (= AES_128). Only relevant for encrypting.

compressionAlgorithm

int

CompressionAlgorithmTags.ZIP

Since camel 2.12.2: compression algorithm; possible values are defined in org.bouncycastle.bcpg.CompressionAlgorithmTags; for example 0 (= UNCOMPRESSED), 1 (= ZIP), 2 (= ZLIB), 3 (= BZIP2). Only relevant for encrypting.

hashAlgorithm

int

HashAlgorithmTags.SHA1

Since camel 2.12.2: signature hash algorithm; possible values are defined in org.bouncycastle.bcpg.HashAlgorithmTags; for example 2 (= SHA1), 8 (= SHA256), 9 (= SHA384), 10 (= SHA512), 11 (=SHA224). Only relevant for signing.

armored

boolean

false

This option will cause PGP to base64 encode the encrypted text, making it available for copy/paste, etc.

integrity

boolean

true

Adds an integrity check/sign into the encryption file.

passphraseAccessor

PGPPassphraseAccessor

null

Since Camel 2.12.2: provides passphrases corresponding to user Ids. If no passpharase can be found from the option password or signaturePassword and from the headers CamelPGPDataFormatKeyPassword or CamelPGPDataFormatSignatureKeyPassword then the passphrase is fetched from the passphrase accessor. You provide a bean which implements the interface PGPPassphraseAccessor. A default implementation is given by DefaultPGPPassphraseAccessor. The passphrase accessor is especially useful in the decrypt case; see chapter 'PGP Decrypting/Verifying of Messages Encrypted/Signed by Different Private/Public Keys' below.

signatureVerificationOption

String

"optional"

Since Camel 2.13.0: controls the behavior for verifying the signature during unmarshaling. There are three values possible:

  • "optional": The PGP message may or may not contain signatures; if it does contain signatures, then a signature verification is executed. Use the constant PGPKeyAccessDataFormat.SIGNATURE_VERIFICATION_OPTION_OPTIONAL.
  • "required": The PGP message must contain at least one signature; if this is not the case an exception (PGPException) is thrown. A signature verification is executed. Use the constant PGPKeyAccessDataFormat.SIGNATURE_VERIFICATION_OPTION_REQUIRED.
  • "ignore": Contained signatures in the PGP message are ignored; no signature verification is executed. Use the constant PGPKeyAccessDataFormat.SIGNATURE_VERIFICATION_OPTION_IGNORE.
  • "no_signature_allowed": The PGP message must not contain a signature; otherwise an exception (PGPException) is thrown. Use the constant PGPKeyAccessDataFormat.SIGNATURE_VERIFICATION_OPTION_NO_SIGNATURE_ALLOWED.

FileName

String"_CONSOLE"

Since camel 2.15.0: Sets the file name for the literal data packet. Can be overwritten by the  header {@link Exchange#FILE_NAME}.

"_CONSOLE" indicates that the message is considered to be "for your eyes only". This advises that the message data is unusually sensitive, and the receiving program should process it more carefully, perhaps avoiding storing the received data to disk, for example.Only used for marshaling.

withCompressedDataPacket

boolean

true

Since Camel 2.16.0: Indicator whether the PGP Message shall be created with or without a Compressed Data packet. If the value is set to false, then no Compressed Data packet is added and the compressionAlgorithm value is ignored. Only used for marshaling.

PGPDataFormat Message Headers

You can override the PGPDataFormat options by applying below headers into message dynamically.

 

Name

Type

Description

CamelPGPDataFormatKeyFileName

String

Since Camel 2.11.0; filename of the keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatEncryptionKeyRing

byte[]

Since Camel 2.12.1; the encryption keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatKeyUserid

String

Since Camel 2.11.0; the User ID of the key in the PGP keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatKeyUserids

List<String>

Since camel 2.12.2: the User IDs of the key in the PGP keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatKeyPassword

String

Since Camel 2.11.0; password used when opening the private key; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureKeyFileName

String

Since Camel 2.11.0; filename of the signature keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureKeyRing

byte[]

Since Camel 2.12.1; the signature keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureKeyUserid

String

Since Camel 2.11.0; the User ID of the signature key in the PGP keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureKeyUseridsList<String>Since Camel 2.12.3; the User IDs of the signature keys in the PGP keyring; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureKeyPassword

String

Since Camel 2.11.0; password used when opening the signature private key; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatEncryptionAlgorithm

int

Since Camel 2.12.2; symmetric key encryption algorithm; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatSignatureHashAlgorithm

int

Since Camel 2.12.2; signature hash algorithm; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatCompressionAlgorithm

int

Since Camel 2.12.2; compression algorithm; will override existing setting directly on the PGPDataFormat.

CamelPGPDataFormatNumberOfEncryptionKeys

IntegerSince Camel 2.12.3;  number of public keys used for encrypting the symmectric key, set by PGPDataFormat during encryptiion process

CamelPGPDataFormatNumberOfSigningKeys

IntegerSince Camel 2.12.3;  number of private keys used for creating signatures, set by PGPDataFormat during signing process

Encrypting with PGPDataFormat

The following sample uses the popular PGP format for encrypting/decrypting files using the Bouncy Castle Java libraries:{snippet:id=pgp-format|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/PGPDataFormatTest.java}The following sample performs signing + encryption, and then signature verification + decryption. It uses the same keyring for both signing and encryption, but you can obviously use different keys:{snippet:id=pgp-format-signature|lang=java|url=camel/trunk/components/camel-crypto/src/test/java/org/apache/camel/converter/crypto/PGPDataFormatTest.java}Or using Spring:{snippet:id=pgp-xml-basic|lang=xml|url=camel/trunk/components/camel-crypto/src/test/resources/org/apache/camel/component/crypto/SpringPGPDataFormatTest.xml}

To work with the previous example you need the following

  • A public keyring file which contains the public keys used to encrypt the data
  • A private keyring file which contains the keys used to decrypt the data
  • The keyring password

Managing your keyring

To manage the keyring, I use the command line tools, I find this to be the simplest approach in managing the keys. There are also Java libraries available from http://www.bouncycastle.org/java.html if you would prefer to do it that way.

  1. Install the command line utilities on linux

    apt-get install gnupg
  2. Create your keyring, entering a secure password

    gpg --gen-key
  3. If you need to import someone elses public key so that you can encrypt a file for them.

    gpg --import <filename.key
  4. The following files should now exist and can be used to run the example

    ls -l ~/.gnupg/pubring.gpg ~/.gnupg/secring.gpg

PGP Decrypting/Verifying of Messages Encrypted/Signed by Different Private/Public Keys

Since Camel 2.12.2.

A PGP Data Formater can decrypt/verify messages which have been encrypted by different public keys or signed by different private keys. Just, provide the corresponding private keys in the secret keyring, the corresponding public keys in the public keyring, and the passphrases in the passphrase accessor.

javaMap<String, String> userId2Passphrase = new HashMap<String, String>(2); // add passphrases of several private keys whose corresponding public keys have been used to encrypt the messages userId2Passphrase.put("UserIdOfKey1","passphrase1"); // you must specify the exact User ID! userId2Passphrase.put("UserIdOfKey2","passphrase2"); PGPPassphraseAccessor passphraseAccessor = new PGPPassphraseAccessorDefault(userId2Passphrase); PGPDataFormat pgpVerifyAndDecrypt = new PGPDataFormat(); pgpVerifyAndDecrypt.setPassphraseAccessor(passphraseAccessor); // the method getSecKeyRing() provides the secret keyring as byte array containing the private keys pgpVerifyAndDecrypt.setEncryptionKeyRing(getSecKeyRing()); // alternatively you can use setKeyFileName(keyfileName) // the method getPublicKeyRing() provides the public keyring as byte array containing the public keys pgpVerifyAndDecrypt.setSignatureKeyRing((getPublicKeyRing()); // alternatively you can use setSignatureKeyFileName(signatgureKeyfileName) // it is not necessary to specify the encryption or signer User Id from("direct:start") ... .unmarshal(pgpVerifyAndDecrypt) // can decrypt/verify messages encrypted/signed by different private/public keys ...
  • The functionality is especially useful to support the key exchange. If you want to exchange the private key for decrypting you can accept for a period of time messages which are either encrypted with the old or new corresponding public key. Or if the sender wants to exchange his signer private key, you can accept for a period of time, the old or new signer key.
  • Technical background: The PGP encrypted data contains a Key ID of the public key which was used to encrypt the data. This Key ID can be used to locate the private key in the secret keyring to decrypt the data. The same mechanism is also used to locate the public key for verifying a signature. Therefore you no longer must specify User IDs for the unmarshaling.

Restricting the Signer Identities during PGP Signature Verification

Since Camel 2.12.3.

If you verify a signature you not only want to verify the correctness of the signature but you also want check that the signature comes from a certain identity or a specific set of identities. Therefore it is possible to restrict the number of public keys from the public keyring which can be used for the verification of a signature.  

javaSignature User IDs// specify the User IDs of the expected signer identities List<String> expectedSigUserIds = new ArrayList<String>(); expectedSigUserIds.add("Trusted company1"); expectedSigUserIds.add("Trusted company2");   PGPDataFormat pgpVerifyWithSpecificKeysAndDecrypt = new PGPDataFormat(); pgpVerifyWithSpecificKeysAndDecrypt.setPassword("my password"); // for decrypting with private key pgpVerifyWithSpecificKeysAndDecrypt.setKeyFileName(keyfileName); pgpVerifyWithSpecificKeysAndDecrypt.setSignatureKeyFileName(signatgureKeyfileName); pgpVerifyWithSpecificKeysAndDecrypt.setSignatureKeyUserids(expectedSigUserIds); // if you have only one signer identity then you can also use setSignatureKeyUserid("expected Signer")   from("direct:start") ... .unmarshal(pgpVerifyWithSpecificKeysAndDecrypt) ...
  • If the PGP content has several signatures the verification is successful as soon as one signature can be verified.
  • If you do not want to restrict the signer identities for verification then do not specify the signature key User IDs. In this case all public keys in the public keyring are taken into account.

Several Signatures in One PGP Data Format

Since Camel 2.12.3.

The PGP specification allows that one PGP data format can contain several signatures from different keys. Since Camel 2.13.3 it is possible to create such kind of PGP content via specifying signature User IDs which relate to several private keys in the secret keyring.

javaSeveral Signatures PGPDataFormat pgpSignAndEncryptSeveralSignerKeys = new PGPDataFormat(); pgpSignAndEncryptSeveralSignerKeys.setKeyUserid(keyUserid); // for encrypting, you can also use setKeyUserids if you want to encrypt with several keys pgpSignAndEncryptSeveralSignerKeys.setKeyFileName(keyfileName); pgpSignAndEncryptSeveralSignerKeys.setSignatureKeyFileName(signatgureKeyfileName); pgpSignAndEncryptSeveralSignerKeys.setSignaturePassword("sdude"); // here we assume that all private keys have the same password, if this is not the case then you can use setPassphraseAccessor List<String> signerUserIds = new ArrayList<String>(); signerUserIds.add("company old key"); signerUserIds.add("company new key"); pgpSignAndEncryptSeveralSignerKeys.setSignatureKeyUserids(signerUserIds);   from("direct:start") ... .marshal(pgpSignAndEncryptSeveralSignerKeys) ...

Support of Sub-Keys and Key Flags in PGP Data Format Marshaler

Since Camel 2.12.3.
An OpenPGP V4 key can have a primary key and sub-keys. The usage of the keys is indicated by the so called Key Flags. For example, you can have a primary key with two sub-keys; the primary key shall only be used for certifying other keys (Key Flag 0x01), the first sub-key  shall only be used for signing (Key Flag 0x02), and the second sub-key shall only be used for encryption (Key Flag 0x04 or 0x08). The PGP Data Format marshaler takes into account these Key Flags of the primary key and sub-keys in order to determine the right key for signing and encryption. This is necessary because the primary key and its sub-keys have the same User IDs.

Support of Custom Key Accessors

Since Camel 2.13.0.
You can implement custom key accessors for encryption/signing. The above PGPDataFormat class selects in a certain predefined way the keys which should be used for signing/encryption or verifying/decryption. If you have special requirements how your keys should be selected you should use the PGPKeyAccessDataFormat class instead and implement the interfaces PGPPublicKeyAccessor and PGPSecretKeyAccessor as beans. There are default implementations DefaultPGPPublicKeyAccessor and DefaultPGPSecretKeyAccessor which cache the keys, so that not every time the keyring is parsed when the processor is called.

PGPKeyAccessDataFormat has the same options as PGPDataFormat except password, keyFileName, encryptionKeyRing, signaturePassword, signatureKeyFileName, and signatureKeyRing.

Dependencies

To use the Crypto dataformat in your camel routes you need to add the following dependency to your pom.

xml<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-crypto</artifactId> <version>x.x.x</version> <!-- use the same version as your Camel core version --> </dependency>

See Also

Syslog DataFormat

Available as of Camel 2.6

The syslog dataformat is used for working with RFC3164 and RFC5424 messages.

This component supports the following:

  • UDP consumption of syslog messages
  • Agnostic data format using either plain String objects or SyslogMessage model objects.
  • Type Converter from/to SyslogMessage and String
  • Integration with the camel-mina component.
  • Integration with the camel-netty component.
  • Camel 2.14: Encoder and decoder for the camel-netty component.
  • Camel 2.14: Support for RFC5424 also.

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-syslog</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>

RFC3164 Syslog protocol

Syslog uses the user datagram protocol (UDP) 1 as its underlying transport layer mechanism.
The UDP port that has been assigned to syslog is 514.

To expose a Syslog listener service we reuse the existing camel-mina component or camel-netty where we just use the Rfc3164SyslogDataFormat to marshal and unmarshal messages. Notice that from Camel 2.14 onwards the syslog dataformat is renamed to SyslogDataFormat.

RFC5424 Syslog protocol

Available as of Camel 2.14

To expose a Syslog listener service we reuse the existing camel-mina component or camel-netty where we just use the SyslogDataFormat to marshal and unmarshal messages

Exposing a Syslog listener

In our Spring XML file, we configure an endpoint to listen for udp messages on port 10514, note that in netty we disable the defaultCodec, this
will allow a fallback to a NettyTypeConverter and delivers the message as an InputStream:

<camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring">

    <dataFormats>
          <syslog id="mySyslog"/>
    </dataFormats>

    <route>
          <from uri="netty:udp://localhost:10514?sync=false&amp;allowDefaultCodec=false"/>
          <unmarshal ref="mySyslog"/>
          <to uri="mock:stop1"/>
    </route>

</camelContext>

The same route using camel-mina

<camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring">

    <dataFormats>
          <syslog id="mySyslog"/>
    </dataFormats>

    <route>
          <from uri="mina:udp://localhost:10514"/>
          <unmarshal ref="mySyslog"/>
          <to uri="mock:stop1"/>
    </route>

</camelContext>

Sending syslog messages to a remote destination

<camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring">

    <dataFormats>
        <syslog id="mySyslog"/>
    </dataFormats>

    <route>
        <from uri="direct:syslogMessages"/>
        <marshal ref="mySyslog"/>
        <to uri="mina:udp://remotehost:10514"/>
    </route>

</camelContext>

Pattern Appendix

There now follows a breakdown of the various Enterprise Integration Patterns that Camel supports

Messaging Systems

Message Channel

Camel supports the Message Channel from the EIP patterns. The Message Channel is an internal implementation detail of the Endpoint interface and all interactions with the Message Channel are via the Endpoint interfaces.


Example

In JMS, Message Channels are represented by topics and queues such as the following

jms:queue:foo

 

This message channel can be then used within the JMS component

Using the Fluent Builders

to("jms:queue:foo")


Using the Spring XML Extensions

<to uri="jms:queue:foo"/>

 

For more details see

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message

Camel supports the Message from the EIP patterns using the Message interface.

To support various message exchange patterns like one way Event Message and Request Reply messages Camel uses an Exchange interface which has a pattern property which can be set to InOnly for an Event Message which has a single inbound Message, or InOut for a Request Reply where there is an inbound and outbound message.

Here is a basic example of sending a Message to a route in InOnly and InOut modes

Requestor Code

//InOnly
getContext().createProducerTemplate().sendBody("direct:startInOnly", "Hello World");

//InOut
String result = (String) getContext().createProducerTemplate().requestBody("direct:startInOut", "Hello World");

Route Using the Fluent Builders

from("direct:startInOnly").inOnly("bean:process");

from("direct:startInOut").inOut("bean:process");

Route Using the Spring XML Extensions

<route>
  <from uri="direct:startInOnly"/>
  <inOnly uri="bean:process"/>
</route>

<route>
  <from uri="direct:startInOut"/>
  <inOut uri="bean:process"/>
</route>

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Pipes and Filters

Camel supports the Pipes and Filters from the EIP patterns in various ways.

With Camel you can split your processing across multiple independent Endpoint instances which can then be chained together.

Using Routing Logic

You can create pipelines of logic using multiple Endpoint or Message Translator instances as follows{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/PipelineTest.java}Though pipeline is the default mode of operation when you specify multiple outputs in Camel. The opposite to pipeline is multicast; which fires the same message into each of its outputs. (See the example below).

In Spring XML you can use the <pipeline/> element

xml<route> <from uri="activemq:SomeQueue"/> <pipeline> <bean ref="foo"/> <bean ref="bar"/> <to uri="activemq:OutputQueue"/> </pipeline> </route>

In the above the pipeline element is actually unnecessary, you could use this:

xml<route> <from uri="activemq:SomeQueue"/> <bean ref="foo"/> <bean ref="bar"/> <to uri="activemq:OutputQueue"/> </route>

which is a bit more explicit.

However if you wish to use <multicast/> to avoid a pipeline - to send the same message into multiple pipelines - then the <pipeline/> element comes into its own:

xml<route> <from uri="activemq:SomeQueue"/> <multicast> <pipeline> <bean ref="something"/> <to uri="log:Something"/> </pipeline> <pipeline> <bean ref="foo"/> <bean ref="bar"/> <to uri="activemq:OutputQueue"/> </pipeline> </multicast> </route>

In the above example we are routing from a single Endpoint to a list of different endpoints specified using URIs. If you find the above a bit confusing, try reading about the Architecture or try the Examples

Using This Pattern

Message Router

The Message Router from the EIP patterns allows you to consume from an input destination, evaluate some predicate then choose the right output destination.

The following example shows how to route a request from an input queue:a endpoint to either queue:b, queue:c or queue:d depending on the evaluation of various Predicate expressions

Using the Fluent Builders

Error formatting macro: snippet: java.lang.NullPointerException

Using the Spring XML Extensions

Error formatting macro: snippet: java.lang.NullPointerException

Choice without otherwise

If you use a choice without adding an otherwise, any unmatched exchanges will be dropped by default.

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Translator

Camel supports the Message Translator from the EIP patterns by using an arbitrary Processor in the routing logic, by using a bean to perform the transformation, or by using transform() in the DSL. You can also use a Data Format to marshal and unmarshal messages in different encodings.

Using the Fluent Builders

You can transform a message using Camel's Bean Integration to call any method on a bean in your Registry such as your Spring XML configuration file as follows

from("activemq:SomeQueue"). beanRef("myTransformerBean", "myMethodName"). to("mqseries:AnotherQueue");

Where the "myTransformerBean" would be defined in a Spring XML file or defined in JNDI etc. You can omit the method name parameter from beanRef() and the Bean Integration will try to deduce the method to invoke from the message exchange.

or you can add your own explicit Processor to do the transformation

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/TransformTest.java}

or you can use the DSL to explicitly configure the transformation

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/TransformProcessorTest.java}

Use Spring XML

You can also use Spring XML Extensions to do a transformation. Basically any Expression language can be substituted inside the transform element as shown below

{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/transformWithExpressionContext.xml}

Or you can use the Bean Integration to invoke a bean

<route> <from uri="activemq:Input"/> <bean ref="myBeanName" method="doTransform"/> <to uri="activemq:Output"/> </route>

You can also use Templating to consume a message from one destination, transform it with something like Velocity or XQuery and then send it on to another destination. For example using InOnly (one way messaging)

from("activemq:My.Queue"). to("velocity:com/acme/MyResponse.vm"). to("activemq:Another.Queue");

If you want to use InOut (request-reply) semantics to process requests on the My.Queue queue on ActiveMQ with a template generated response, then sending responses back to the JMSReplyTo Destination you could use this.

from("activemq:My.Queue"). to("velocity:com/acme/MyResponse.vm"); Using This Pattern

Message Endpoint

Camel supports the Message Endpoint from the EIP patterns using the Endpoint interface.

When using the DSL to create Routes you typically refer to Message Endpoints by their URIs rather than directly using the Endpoint interface. Its then a responsibility of the CamelContext to create and activate the necessary Endpoint instances using the available Component implementations.

Example

The following example route demonstrates the use of a File Consumer Endpoint and JMS Producer Endpoint


Using the Fluent Builders

from("file://local/router/messages/foo")
	.to("jms:queue:foo");

 

Using the Spring XML Extensions

<route>
	<from uri="file://local/router/messages/foo"/>
	<to uri="jms:queue:foo"/>
</route>

 

Dynamic To

Available as of Camel 2.16

There is a new <toD> that allows to send a message to a dynamic computed Endpoint using one or more Expression that are concat together. By default the Simple language is used to compute the endpoint. For example to send a message to a endpoint defined by a header you can do

<route>
  <from uri="direct:start"/>
  <toD uri="${header.foo}"/>
</route>

And in Java DSL

from("direct:start")
  .toD("${header.foo}");

 

You can also prefix the uri with a value because by default the uri is evaluated using the Simple language

<route>
  <from uri="direct:start"/>
  <toD uri="mock:${header.foo}"/>
</route>

And in Java DSL

from("direct:start")
  .toD("mock:${header.foo}");

In the example above we compute an endpoint that has prefix "mock:" and then the header foo is appended. So for example if the header foo has value order, then the endpoint is computed as "mock:order".

You can also use other languages than Simple such as XPath - this requires to prefix with language: as shown below (simple language is the default language). If you do not specify language: then the endpoint is a component name. And in some cases there is both a component and language with the same name such as xquery.

<route>
  <from uri="direct:start"/>
  <toD uri="language:xpath:/order/@uri"/>
</route>

This is done by specifying the name of the language followed by a colon.

from("direct:start")
  .toD("language:xpath:/order/@uri");

You can also concat multiple Language(s) together using the plus sign + such as shown below:

<route>
  <from uri="direct:start"/>
  <toD uri="jms:${header.base}+language:xpath:/order/@id"/>
</route>

In the example above the uri is a combination of Simple language and XPath where the first part is simple (simple is default language). And then the plus sign separate to another language, where we specify the language name followed by a colon

from("direct:start")
  .toD("jms:${header.base}+language:xpath:/order/@id");

You can concat as many languages as you want, just separate them with the plus sign

The Dynamic To has a few options you can configure

NameDefault ValueDescription
uri Mandatory: The uri to use. See above
pattern To set a specific Exchange Pattern to use when sending to the endpoint. The original MEP is restored afterwards.
cacheSize Allows to configure the cache size for the ProducerCache which caches producers for reuse. Will by default use the default cache size which is 1000. Setting the value to -1 allows to turn off the cache all together.
ignoreInvalidEndpointfalseWhether to ignore an endpoint URI that could not be resolved. If disabled, Camel will throw an exception identifying the invalid endpoint URI.

 

For more details see

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Messaging Channels

Error rendering macro 'include'

null

Publish Subscribe Channel

Camel supports the Publish Subscribe Channel from the EIP patterns using for example the following components:

  • JMS for working with JMS Topics for high performance, clustering and load balancing
  • XMPP when using rooms for group communication
  • SEDA for working with SEDA in the same CamelContext which can work in pub-sub, but allowing multiple consumers.
  • VM as SEDA but for intra-JVM.

Using Routing Logic

Another option is to explicitly list the publish-subscribe relationship in your routing logic; this keeps the producer and consumer decoupled but lets you control the fine grained routing configuration using the DSL or Xml Configuration.

Using the Fluent Builders

Error formatting macro: snippet: java.lang.NullPointerException

Using the Spring XML Extensions

Error formatting macro: snippet: java.lang.NullPointerException

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Dead Letter Channel

Camel supports the Dead Letter Channel from the EIP patterns using the DeadLetterChannel processor which is an Error Handler.

Differences Between The DeadLetterChannel And The DefaultErrorHandler

The DefaultErrorHandler does very little: it ends the Exchange immediately and propagates the thrown Exception back to the caller.

The DeadLetterChannel lets you control behaviors including redelivery, whether to propagate the thrown Exception to the caller (the handled option), and where the (failed) Exchange should now be routed to.

The DeadLetterChannel is also by default configured to not be verbose in the logs, so when a message is handled and moved to the dead letter endpoint, then there is nothing logged. If you want some level of logging you can use the various options on the redelivery policy / dead letter channel to configure this. For example if you want the message history then set logExhaustedMessageHistory=true (and logHandled=true for Camel 2.15.x or older).

When the DeadLetterChannel moves a message to the dead letter endpoint, any new Exception thrown is by default handled by the dead letter channel as well. This ensures that the DeadLetterChannel will always succeed. From Camel 2.15: this behavior can be changed by setting the option deadLetterHandleNewException=false. Then if a new Exception is thrown, then the dead letter channel will fail and propagate back that new Exception (which is the behavior of the default error handler). When a new Exception occurs then the dead letter channel logs this at WARN level. This can be turned off by setting logNewException=false.

Redelivery

It is common for a temporary outage or database deadlock to cause a message to fail to process; but the chances are if its tried a few more times with some time delay then it will complete fine. So we typically wish to use some kind of redelivery policy to decide how many times to try redeliver a message and how long to wait before redelivery attempts.

The RedeliveryPolicy defines how the message is to be redelivered. You can customize things like

  • The number of times a message is attempted to be redelivered before it is considered a failure and sent to the dead letter channel.
  • The initial redelivery timeout.
  • Whether or not exponential backoff is used, i.e., the time between retries increases using a backoff multiplier.
  • Whether to use collision avoidance to add some randomness to the timings.
  • Delay pattern (see below for details).
  • Camel 2.11: Whether to allow redelivery during stopping/shutdown.

Once all attempts at redelivering the message fails then the message is forwarded to the dead letter queue.

About Moving Exchange to Dead Letter Queue and Using handled()

handled() on Dead Letter Channel

When all attempts of redelivery have failed the Exchange is moved to the dead letter queue (the dead letter endpoint). The exchange is then complete and from the client point of view it was processed. As such the Dead Letter Channel have handled the Exchange.

For instance configuring the dead letter channel as:

Using the Fluent Builders

javaerrorHandler(deadLetterChannel("jms:queue:dead") .maximumRedeliveries(3).redeliveryDelay(5000));

Using the Spring XML Extensions

xml<route errorHandlerRef="myDeadLetterErrorHandler"> <!-- ... --> </route> <bean id="myDeadLetterErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder"> <property name="deadLetterUri" value="jms:queue:dead"/> <property name="redeliveryPolicy" ref="myRedeliveryPolicyConfig"/> </bean> <bean id="myRedeliveryPolicyConfig" class="org.apache.camel.processor.RedeliveryPolicy"> <property name="maximumRedeliveries" value="3"/> <property name="redeliveryDelay" value="5000"/> </bean>

The Dead Letter Channel above will clear the caused exception setException(null), by moving the caused exception to a property on the Exchange, with the key Exchange.EXCEPTION_CAUGHT. Then the Exchange is moved to the jms:queue:dead destination and the client will not notice the failure.

About Moving Exchange to Dead Letter Queue and Using the Original Message

The option useOriginalMessage is used for routing the original input message instead of the current message that potentially is modified during routing.

For instance if you have this route:

java from("jms:queue:order:input") .to("bean:validateOrder") .to("bean:transformOrder") .to("bean:handleOrder");

The route listen for JMS messages and validates, transforms and handle it. During this the Exchange payload is transformed/modified. So in case something goes wrong and we want to move the message to another JMS destination, then we can configure our Dead Letter Channel with the useOriginalMessage option. But when we move the Exchange to this destination we do not know in which state the message is in. Did the error happen in before the transformOrder or after? So to be sure we want to move the original input message we received from jms:queue:order:input. So we can do this by enabling the useOriginalMessage option as shown below:

java// will use original body errorHandler(deadLetterChannel("jms:queue:dead") .useOriginalMessage() .maximumRedeliveries(5) .redeliverDelay(5000);

Then the messages routed to the jms:queue:dead is the original input. If we want to manually retry we can move the JMS message from the failed to the input queue, with no problem as the message is the same as the original we received.

OnRedelivery

When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered. See below for sample.

onException and onRedeliver

We also support for per onException to set an onRedeliver. That means you can do special on redelivery for different exceptions, as opposed to onRedelivery set on Dead Letter Channel can be viewed as a global scope.

Redelivery Default Values

Redelivery is disabled by default.

The default redeliver policy will use the following values:

  • maximumRedeliveries=0
  • redeliverDelay=1000L (1 second)
  • maximumRedeliveryDelay = 60 * 1000L (60 seconds)
  • backOffMultiplier and useExponentialBackOff are ignored.
  • retriesExhaustedLogLevel=LoggingLevel.ERROR
  • retryAttemptedLogLevel=LoggingLevel.DEBUG
  • Stack traces are logged for exhausted messages, from Camel 2.2.
  • Handled exceptions are not logged, from Camel 2.3.
  • logExhaustedMessageHistory is true for default error handler, and false for dead letter channel.
  • logExhaustedMessageBody Camel 2.17: is disabled by default to avoid logging sensitive message body/header details. If this option is true, then logExhaustedMessageHistory must also be true.

The maximum redeliver delay ensures that a delay is never longer than the value, default 1 minute. This can happen when useExponentialBackOff=true.

The maximumRedeliveries is the number of re-delivery attempts. By default Camel will try to process the exchange 1 + 5 times. 1 time for the normal attempt and then 5 attempts as redeliveries.
Setting the maximumRedeliveries=-1 (or < -1) will then always redelivery (unlimited).
Setting the maximumRedeliveries=0 will disable re-delivery.

Camel will log delivery failures at the DEBUG logging level by default. You can change this by specifying retriesExhaustedLogLevel and/or retryAttemptedLogLevel. See ExceptionBuilderWithRetryLoggingLevelSetTest for an example.

You can turn logging of stack traces on/off. If turned off Camel will still log the redelivery attempt. It's just much less verbose.

Redeliver Delay Pattern

Delay pattern is used as a single option to set a range pattern for delays. When a delay pattern is in use the following options no longer apply:

  • delay
  • backOffMultiplier
  • useExponentialBackOff
  • useCollisionAvoidance
  • maximumRedeliveryDelay

The idea is to set groups of ranges using the following syntax: limit:delay;limit 2:delay 2;limit 3:delay 3;...;limit N:delay N

Each group has two values separated with colon:

  • limit = upper limit
  • delay = delay in milliseconds
    And the groups is again separated with semi-colon. The rule of thumb is that the next groups should have a higher limit than the previous group.

Lets clarify this with an example:
delayPattern=5:1000;10:5000;20:20000

That gives us three groups:

  • 5:1000
  • 10:5000
  • 20:20000

Resulting in these delays between redelivery attempts:

  • Redelivery attempt number 1..4 = 0ms (as the first group start with 5)
  • Redelivery attempt number 5..9 = 1000ms (the first group)
  • Redelivery attempt number 10..19 = 5000ms (the second group)
  • Redelivery attempt number 20.. = 20000ms (the last group)

Note: The first redelivery attempt is 1, so the first group should start with 1 or higher.

You can start a group with limit 1 to e.g., have a starting delay: delayPattern=1:1000;5:5000

  • Redelivery attempt number 1..4 = 1000ms (the first group)
  • Redelivery attempt number 5.. = 5000ms (the last group)

There is no requirement that the next delay should be higher than the previous. You can use any delay value you like. For example with delayPattern=1:5000;3:1000 we start with 5 sec delay and then later reduce that to 1 second.

Redelivery header

When a message is redelivered the DeadLetterChannel will append a customizable header to the message to indicate how many times its been redelivered.
Before Camel 2.6: The header is CamelRedeliveryCounter, which is also defined on the Exchange.REDELIVERY_COUNTER.
From Camel 2.6: The header CamelRedeliveryMaxCounter, which is also defined on the Exchange.REDELIVERY_MAX_COUNTER, contains the maximum redelivery setting. This header is absent if you use retryWhile or have unlimited maximum redelivery configured.

And a boolean flag whether it is being redelivered or not (first attempt). The header CamelRedelivered contains a boolean if the message is redelivered or not, which is also defined on the Exchange.REDELIVERED.

Dynamically Calculated Delay From the Exchange

In Camel 2.9 and 2.8.2: The header is CamelRedeliveryDelay, which is also defined on the Exchange.REDELIVERY_DELAY. If this header is absent, normal redelivery rules apply.

Which Endpoint Failed

Available as of Camel 2.1

When Camel routes messages it will decorate the Exchange with a property that contains the last endpoint Camel send the Exchange to:

javaString lastEndpointUri = exchange.getProperty(Exchange.TO_ENDPOINT, String.class);

The Exchange.TO_ENDPOINT have the constant value CamelToEndpoint. This information is updated when Camel sends a message to any endpoint. So if it exists its the last endpoint which Camel send the Exchange to.

When for example processing the Exchange at a given Endpoint and the message is to be moved into the dead letter queue, then Camel also decorates the Exchange with another property that contains that last endpoint:

javaString failedEndpointUri = exchange.getProperty(Exchange.FAILURE_ENDPOINT, String.class);

The Exchange.FAILURE_ENDPOINT have the constant value CamelFailureEndpoint.

This allows for example you to fetch this information in your dead letter queue and use that for error reporting. This is usable if the Camel route is a bit dynamic such as the dynamic Recipient List so you know which endpoints failed.

Note: this information is retained on the Exchange even if the message is subsequently processed successfully by a given endpoint only to fail, for example, in local Bean processing instead. So, beware that this is a hint that helps pinpoint errors.

javafrom("activemq:queue:foo") .to("http://someserver/somepath") .beanRef("foo");

Now suppose the route above and a failure happens in the foo bean. Then the Exchange.TO_ENDPOINT and Exchange.FAILURE_ENDPOINT will still contain the value of http://someserver/somepath.

OnPrepareFailure

Available as of Camel 2.16

Before the exchange is sent to the dead letter queue, you can use onPrepare to allow a custom Processor to prepare the exchange, such as adding information why the Exchange failed.

For example, the following processor adds a header with the exception message:

java public static class MyPrepareProcessor implements Processor { @Override public void process(Exchange exchange) throws Exception { Exception cause = exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class); exchange.getIn().setHeader("FailedBecause", cause.getMessage()); } }

Then configure the error handler to use the processor as follows:

javaerrorHandler(deadLetterChannel("jms:dead").onPrepareFailure(new MyPrepareProcessor()));

 

Configuring this from XML DSL is as follows:

xml<bean id="myPrepare" class="org.apache.camel.processor.DeadLetterChannelOnPrepareTest.MyPrepareProcessor"/> <errorHandler id="dlc" type="DeadLetterChannel" deadLetterUri="jms:dead" onPrepareFailureRef="myPrepare"/>

 

The onPrepare is also available using the default error handler.

Which Route Failed

Available as of Camel 2.10.4/2.11

When Camel error handler handles an error such as Dead Letter Channel or using Exception Clause with handled=true, then Camel will decorate the Exchange with the route id where the error occurred.

Example:

javaString failedRouteId = exchange.getProperty(Exchange.FAILURE_ROUTE_ID, String.class);

The Exchange.FAILURE_ROUTE_ID have the constant value CamelFailureRouteId. This allows for example you to fetch this information in your dead letter queue and use that for error reporting.

Control if Redelivery is Allowed During Stopping/Shutdown

Available as of Camel 2.11

Before Camel 2.10, Camel would perform redelivery while stopping a route, or shutting down Camel. This has improved a bit in Camel 2.10: Camel will no longer perform redelivery attempts when shutting down aggressively, e.g., during Graceful Shutdown and timeout hit.

From Camel 2.11: there is a new option allowRedeliveryWhileStopping which you can use to control if redelivery is allowed or not; notice that any in progress redelivery will still be executed. This option can only disallow any redelivery to be executed after the stopping of a route/shutdown of Camel has been triggered. If a redelivery is disallowed then a RejectedExcutionException is set on the Exchange and the processing of the Exchange stops. This means any consumer will see the Exchange as failed due the RejectedExcutionException. The default value is true for backward compatibility.

For example, the following snippet shows how to do this with Java DSL and XML DSL:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RedeliveryErrorHandlerNoRedeliveryOnShutdownTest.java}And the sample sample with XML DSL{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringRedeliveryErrorHandlerNoRedeliveryOnShutdownTest.xml}

Samples

The following example shows how to configure the Dead Letter Channel configuration using the DSL{snippet:id=e3|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/ErrorHandlerTest.java}You can also configure the RedeliveryPolicy as this example shows{snippet:id=e4|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/ErrorHandlerTest.java}

How Can I Modify the Exchange Before Redelivery?

We support directly in Dead Letter Channel to set a Processor that is executed before each redelivery attempt. When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered. Here we configure the Dead Letter Channel to use our processor MyRedeliveryProcessor to be executed before each redelivery.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DeadLetterChannelOnRedeliveryTest.java}And this is the processor MyRedeliveryProcessor where we alter the message.{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DeadLetterChannelOnRedeliveryTest.java}

How Can I Log What Caused the Dead Letter Channel to be Invoked?

You often need to know what went wrong that caused the Dead Letter Channel to be used and it does not offer logging for this purpose. So the Dead Letter Channel's endpoint can be set to a endpoint of our own (such as direct:deadLetterChannel). We write a route to accept this Exchange and log the Exception, then forward on to where we want the failed Exchange moved to (which might be a DLQ queue for instance). See also http://stackoverflow.com/questions/13711462/logging-camel-exceptions-and-sending-to-the-dead-letter-channel

Using This Pattern

Guaranteed Delivery

Camel supports the Guaranteed Delivery from the EIP patterns using among others the following components:

  • File for using file systems as a persistent store of messages
  • JMS when using persistent delivery (the default) for working with JMS Queues and Topics for high performance, clustering and load balancing
  • JPA for using a database as a persistence layer, or use any of the many other database component such as SQL, JDBC, iBATIS/MyBatis, Hibernate
  • HawtDB for a lightweight key-value persistent store

Example

The following example demonstrates illustrates the use of Guaranteed Delivery within the JMS component. By default, a message is not considered successfully delivered until the recipient has persisted the message locally guaranteeing its receipt in the event the destination becomes unavailable.

Using the Fluent Builders

from("direct:start")
	.to("jms:queue:foo");

 

Using the Spring XML Extensions

<route>
	<from uri="direct:start"/>
	<to uri="jms:queue:foo"/>
</route>

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Bus

Camel supports the Message Bus from the EIP patterns. You could view Camel as a Message Bus itself as it allows producers and consumers to be decoupled.

Folks often assume that a Message Bus is a JMS though so you may wish to refer to the JMS component for traditional MOM support.
Also worthy of note is the XMPP component for supporting messaging over XMPP (Jabber)

Of course there are also ESB products such as Apache ServiceMix which serve as full fledged message busses.
You can interact with Apache ServiceMix from Camel in many ways, but in particular you can use the NMR or JBI component to access the ServiceMix message bus directly.

 

Example

The following demonstrates how the Camel message bus can be used to communicate with consumers and producers


Using the Fluent Builders

from("direct:start")
	.pollEnrich("file:inbox?fileName=data.txt")
	.to("jms:queue:foo");

 

Using the Spring XML Extensions

<route>
	<from uri="direct:start"/>
	<pollEnrich uri="file:inbox?fileName=data.txt"/>
	<to uri="jms:queue:foo"/>
</route>

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Construction

Event Message

Camel supports the Event Message from the EIP patterns by supporting the Exchange Pattern on a Message which can be set to InOnly to indicate a oneway event message. Camel Components then implement this pattern using the underlying transport or protocols.

The default behaviour of many Components is InOnly such as for JMS, File or SEDA

Related

See the related Request Reply message.

Explicitly specifying InOnly

If you are using a component which defaults to InOut you can override the Exchange Pattern for an endpoint using the pattern property.

foo:bar?exchangePattern=InOnly

From 2.0 onwards on Camel you can specify the Exchange Pattern using the DSL.

Using the Fluent Builders

from("mq:someQueue").
  setExchangePattern(ExchangePattern.InOnly).
  bean(Foo.class);

or you can invoke an endpoint with an explicit pattern

from("mq:someQueue").
  inOnly("mq:anotherQueue");

Using the Spring XML Extensions

<route>
    <from uri="mq:someQueue"/>
    <inOnly uri="bean:foo"/>
</route>
<route>
    <from uri="mq:someQueue"/>
    <inOnly uri="mq:anotherQueue"/>
</route>

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Error rendering macro 'include'

org.owasp.validator.html.ScanException: java.util.EmptyStackException

Correlation Identifier

Camel supports the Correlation Identifier from the EIP patterns by getting or setting a header on a Message.

When working with the ActiveMQ or JMS components the correlation identifier header is called JMSCorrelationID. You can add your own correlation identifier to any message exchange to help correlate messages together to a single conversation (or business process).

The use of a Correlation Identifier is key to working with the Camel Business Activity Monitoring Framework and can also be highly useful when testing with simulation or canned data such as with the Mock testing framework

Some EIP patterns will spin off a sub message, and in those cases, Camel will add a correlation id on the Exchange as a property with they key Exchange.CORRELATION_ID, which links back to the source Exchange. For example the Splitter, Multicast, Recipient List, and Wire Tap EIP does this.

The following example demonstrates using the Camel JMSMessageID as the Correlation Identifier within a request/reply pattern in the JMS component

Using the Fluent Builders

from("direct:start")
	.to(ExchangePattern.InOut,"jms:queue:foo?useMessageIDAsCorrelationID=true")
	.to("mock:result");

 

Using the Spring XML Extensions

<route>
	<from uri="direct:start"/>
	<to uri="jms:queue:foo?useMessageIDAsCorrelationID=true" pattern="InOut"/>
	<to uri="mock:result"/>
</route>

See Also

Return Address

Camel supports the Return Address from the EIP patterns by using the JMSReplyTo header.

For example when using JMS with InOut the component will by default return to the address given in JMSReplyTo.

Requestor Code

getMockEndpoint("mock:bar").expectedBodiesReceived("Bye World");
template.sendBodyAndHeader("direct:start", "World", "JMSReplyTo", "queue:bar");

Route Using the Fluent Builders

from("direct:start").to("activemq:queue:foo?preserveMessageQos=true");
from("activemq:queue:foo").transform(body().prepend("Bye "));
from("activemq:queue:bar?disableReplyTo=true").to("mock:bar");

Route Using the Spring XML Extensions

<route>
  <from uri="direct:start"/>
  <to uri="activemq:queue:foo?preserveMessageQos=true"/>
</route>

<route>
  <from uri="activemq:queue:foo"/>
  <transform>
      <simple>Bye ${in.body}</simple>
  </transform>
</route>

<route>
  <from uri="activemq:queue:bar?disableReplyTo=true"/>
  <to uri="mock:bar"/>
</route>

For a complete example of this pattern, see this junit test case

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Routing

Content Based Router

The Content Based Router from the EIP patterns allows you to route messages to the correct destination based on the contents of the message exchanges.

The following example shows how to route a request from an input seda:a endpoint to either seda:b, seda:c or seda:d depending on the evaluation of various Predicate expressions

Using the Fluent Builders

 


RouteBuilder builder = new RouteBuilder() {
    public void configure() {
        errorHandler(deadLetterChannel("mock:error"));
 
        from("direct:a")
            .choice()
                .when(header("foo").isEqualTo("bar"))
                    .to("direct:b")
                .when(header("foo").isEqualTo("cheese"))
                    .to("direct:c")
                .otherwise()
                    .to("direct:d");
    }
};

See Why can I not use when or otherwise in a Java Camel route if you have problems with the Java DSL, accepting using when or otherwise.

Using the Spring XML Extensions

 

<camelContext errorHandlerRef="errorHandler" xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="direct:a"/>
        <choice>
            <when>
                <xpath>$foo = 'bar'</xpath>
                <to uri="direct:b"/>
            </when>
            <when>
                <xpath>$foo = 'cheese'</xpath>
                <to uri="direct:c"/>
            </when>
            <otherwise>
                <to uri="direct:d"/>
            </otherwise>
        </choice>
    </route>
</camelContext>

For further examples of this pattern in use you could look at the junit test case

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Filter

The Message Filter from the EIP patterns allows you to filter messages

The following example shows how to create a Message Filter route consuming messages from an endpoint called queue:a, which if the Predicate is true will be dispatched to queue:b

Using the Fluent Builders{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/RouteBuilderTest.java}You can, of course, use many different Predicate languages such as XPath, XQuery, SQL or various Scripting Languages. Here is an XPath example{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/XPathFilterTest.java}Here's another example of using a bean to define the filter behavior:

javafrom("direct:start") .filter().method(MyBean.class, "isGoldCustomer").to("mock:result").end() .to("mock:end"); public static class MyBean { public boolean isGoldCustomer(@Header("level") String level) { return level.equals("gold"); } }

Using the Spring XML Extensions{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/xml/buildSimpleRouteWithHeaderPredicate.xml}You can also use a method call expression (to call a method on a bean) in the Message Filter, as shown below:

xml<bean id="myBean" class="com.foo.MyBean"/> <camelContext xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:a"/> <filter> <method ref="myBean" method="isGoldCustomer"/> <to uri="direct:b"/> </filter> </route> </camelContext>Filtered Endpoint Required Inside </filter> Tag

Ensure you put the endpoint you want to filter <to uri="seda:b"/> before the closing </filter> tag or the filter will not be applied. From Camel 2.8: omitting this will result in an error.

For further examples of this pattern in use you could look at the junit test case

Using stop()

Stop is a bit different than a message filter as it will filter out all messages and end the route entirely (filter only applies to its child processor). Stop is convenient to use in a Content Based Router when you for example need to stop further processing in one of the predicates.

In the example below we do not want to route messages any further that has the word Bye in the message body. Notice how we prevent this in the when() predicate by using the .stop().{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RouteStopTest.java}

How To Determine If An  Exchange Was Filtered

Available as of Camel 2.5

The Message Filter EIP will add a property on the Exchange that states if it was filtered or not.

The property has the key Exchange.FILTER_MATCHED, which has the String value of CamelFilterMatched. Its value is a boolean indicating true or false. If the value is true then the Exchange was routed in the filter block. This property will be visible within the Message Filter block who's Predicate matches (value set to true), and to the steps immediately following the Message Filter with the value set based on the results of the last Message Filter Predicate evaluated.

Using This Pattern

Dynamic Router

The Dynamic Router from the EIP patterns allows you to route messages while avoiding the dependency of the router on all possible destinations while maintaining its efficiency.

In Camel 2.5 we introduced a dynamicRouter in the DSL which is like a dynamic Routing Slip which evaluates the slip on-the-fly.

Beware

You must ensure the expression used for the dynamicRouter such as a bean, will return null to indicate the end. Otherwise the dynamicRouter will keep repeating endlessly.

Options

confluenceTableSmall

Name

Default Value

Description

uriDelimiter

,

Delimiter used if the Expression returned multiple endpoints.

ignoreInvalidEndpoints

false

If an endpoint uri could not be resolved, should it be ignored. Otherwise Camel will thrown an exception stating the endpoint uri is not valid.

cacheSize

1000

Camel 2.13.1/2.12.4: Allows to configure the cache size for the ProducerCache which caches producers for reuse in the routing slip. Will by default use the default cache size which is 1000. Setting the value to -1 allows to turn off the cache all together.

Dynamic Router in Camel 2.5 onwards

From Camel 2.5 the Dynamic Router will set a property (Exchange.SLIP_ENDPOINT) on the Exchange which contains the current endpoint as it advanced though the slip. This allows you to know how far we have processed in the slip. (It's a slip because the Dynamic Router implementation is based on top of Routing Slip).

Java DSL

In Java DSL you can use the dynamicRouter as shown below:

{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DynamicRouterTest.java}

Which will leverage a Bean to compute the slip on-the-fly, which could be implemented as follows:

{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DynamicRouterTest.java}

Mind that this example is only for show and tell. The current implementation is not thread safe. You would have to store the state on the Exchange, to ensure thread safety, as shown below:

{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DynamicRouterExchangePropertiesTest.java}

You could also store state as message headers, but they are not guaranteed to be preserved during routing, where as properties on the Exchange are. Although there was a bug in the method call expression, see the warning below.

Using beans to store state

Mind that in Camel 2.9.2 or older, when using a Bean the state is not propagated, so you will have to use a Processor instead. This is fixed in Camel 2.9.3 onwards.

Spring XML

The same example in Spring XML would be:

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringDynamicRouterTest.xml}

@DynamicRouter annotation

You can also use the @DynamicRouter annotation, for example the Camel 2.4 example below could be written as follows. The route method would then be invoked repeatedly as the message is processed dynamically. The idea is to return the next endpoint uri where to go. Return null to indicate the end. You can return multiple endpoints if you like, just as the Routing Slip, where each endpoint is separated by a delimiter.

javapublic class MyDynamicRouter { @Consume(uri = "activemq:foo") @DynamicRouter public String route(@XPath("/customer/id") String customerId, @Header("Location") String location, Document body) { // query a database to find the best match of the endpoint based on the input parameteres // return the next endpoint uri, where to go. Return null to indicate the end. } }

Dynamic Router in Camel 2.4 or older

The simplest way to implement this is to use the RecipientList Annotation on a Bean method to determine where to route the message.

javapublic class MyDynamicRouter { @Consume(uri = "activemq:foo") @RecipientList public List<String> route(@XPath("/customer/id") String customerId, @Header("Location") String location, Document body) { // query a database to find the best match of the endpoint based on the input parameteres ... } }

In the above we can use the Parameter Binding Annotations to bind different parts of the Message to method parameters or use an Expression such as using XPath or XQuery.

The method can be invoked in a number of ways as described in the Bean Integration such as

Using This Pattern

Error rendering macro 'include'

null

Error rendering macro 'include'

null

Aggregator

This applies for Camel version 2.3 or newer. If you use an older version then use this Aggregator link instead.

The Aggregator from the EIP patterns allows you to combine a number of messages together into a single message.

A correlation Expression is used to determine the messages which should be aggregated together. If you want to aggregate all messages into a single message, just use a constant expression. An AggregationStrategy is used to combine all the message exchanges for a single correlation key into a single message exchange.

Aggregator options

The aggregator supports the following options:

confluenceTableSmall

Option

Default

Description

correlationExpression

 

Mandatory Expression which evaluates the correlation key to use for aggregation. The Exchange which has the same correlation key is aggregated together. If the correlation key could not be evaluated an Exception is thrown. You can disable this by using the ignoreBadCorrelationKeys option.

aggregationStrategy

 

Mandatory AggregationStrategy which is used to merge the incoming Exchange with the existing already merged exchanges. At first call the oldExchange parameter is null. On subsequent invocations the oldExchange contains the merged exchanges and newExchange is of course the new incoming Exchange.

From Camel 2.9.2 onwards the strategy can also be a TimeoutAwareAggregationStrategy implementation, supporting the timeout callback, see further below for more details.

From Camel 2.16: the strategy can also be a PreCompletionAwareAggregationStrategy implementation which then runs the completion check in pre-completion mode. See further below for more details.

strategyRef

 

A reference to lookup the AggregationStrategy in the Registry. From Camel 2.12 onwards you can also use a POJO as the AggregationStrategy, see further below for details.

strategyMethodName

 

Camel 2.12: This option can be used to explicit declare the method name to use, when using POJOs as the AggregationStrategy. See further below for more details.

strategyMethodAllowNull

false

Camel 2.12: If this option is false then the aggregate method is not used for the very first aggregation. If this option is true then null values is used as the oldExchange (at the very first aggregation), when using POJOs as the AggregationStrategy. See further below for more details.

completionSize

 

Number of messages aggregated before the aggregation is complete. This option can be set as either a fixed value or using an Expression which allows you to evaluate a size dynamically - will use Integer as result. If both are set Camel will fallback to use the fixed value if the Expression result was null or 0.

completionTimeout

 

Time in millis that an aggregated exchange should be inactive before its complete. This option can be set as either a fixed value or using an Expression which allows you to evaluate a timeout dynamically - will use Long as result. If both are set Camel will fallback to use the fixed value if the Expression result was null or 0. You cannot use this option together with completionInterval, only one of the two can be used.

completionInterval

 

A repeating period in millis by which the aggregator will complete all current aggregated exchanges. Camel has a background task which is triggered every period. You cannot use this option together with completionTimeout, only one of them can be used.

completionPredicate

 

A Predicate to indicate when an aggregated exchange is complete.

From Camel 2.15: if this is not specified and the AggregationStrategy object implements Predicate, the aggregationStrategy object will be used as the completionPredicate.

completionFromBatchConsumer

false

This option is if the exchanges are coming from a Batch Consumer. Then when enabled the Aggregator2 will use the batch size determined by the Batch Consumer in the message header CamelBatchSize. See more details at Batch Consumer. This can be used to aggregate all files consumed from a File endpoint in that given poll.

forceCompletionOnStop

false

Camel 2.9 Indicates to complete all current aggregated exchanges when the context is stopped

completeAllOnStopfalseCamel 2.16: Indicates to wait to complete all current and partial (pending) aggregated exchanges when the context is stopped. This also means that we will wait for all pending exchanges which are stored in the aggregation repository to complete so the repository is empty before we can stop.  You may want to enable this when using the memory based aggregation repository that is memory based only, and do not store data on disk. When this option is enabled, then the aggregator is waiting to complete all those exchanges before its stopped, when stopping CamelContext or the route using it.

eagerCheckCompletion

false

Whether or not to eager check for completion when a new incoming Exchange has been received. This option influences the behavior of the completionPredicate option as the Exchange being passed in changes accordingly. When false the Exchange passed in the Predicate is the aggregated Exchange which means any information you may store on the aggregated Exchange from the AggregationStrategy is available for the Predicate. When true the Exchange passed in the Predicate is the incoming Exchange, which means you can access data from the incoming Exchange.

groupExchanges

false

If enabled then Camel will group all aggregated Exchanges into a single combined org.apache.camel.impl.GroupedExchange holder class that holds all the aggregated Exchanges. And as a result only one Exchange is being sent out from the aggregator. Can be used to combine many incoming Exchanges into a single output Exchange without coding a custom AggregationStrategy yourself.

Note: this option does not support persistent repository with the aggregator. See further below for an example and more details.

ignoreInvalidCorrelationKeys

false

Whether or not to ignore correlation keys which could not be evaluated to a value. By default Camel will throw an Exception, but you can enable this option and ignore the situation instead.

closeCorrelationKeyOnCompletion

 

Whether or not too late Exchanges should be accepted or not. You can enable this to indicate that if a correlation key has already been completed, then any new exchanges with the same correlation key be denied. Camel will then throw a closedCorrelationKeyException exception. When using this option you pass in a integer which is a number for a LRUCache which keeps that last X number of closed correlation keys. You can pass in 0 or a negative value to indicate a unbounded cache. By passing in a number you are ensured that cache won't grow too big if you use a log of different correlation keys.

discardOnCompletionTimeout

false

Camel 2.5: Whether or not exchanges which complete due to a timeout should be discarded. If enabled then when a timeout occurs the aggregated message will not be sent out but dropped (discarded).

aggregationRepository

 

Allows you to plugin you own implementation of org.apache.camel.spi.AggregationRepository which keeps track of the current inflight aggregated exchanges. Camel uses by default a memory based implementation.

aggregationRepositoryRef

 

Reference to lookup a aggregationRepository in the Registry.

parallelProcessing

false

When aggregated are completed they are being send out of the aggregator. This option indicates whether or not Camel should use a thread pool with multiple threads for concurrency. If no custom thread pool has been specified then Camel creates a default pool with 10 concurrent threads.

executorService

 

If using parallelProcessing you can specify a custom thread pool to be used. In fact also if you are not using parallelProcessing this custom thread pool is used to send out aggregated exchanges as well.

executorServiceRef

 

Reference to lookup a executorService in the Registry

timeoutCheckerExecutorService

 

Camel 2.9: If using either of the completionTimeout, completionTimeoutExpression, or completionInterval options a background thread is created to check for the completion for every aggregator. Set this option to provide a custom thread pool to be used rather than creating a new thread for every aggregator.

timeoutCheckerExecutorServiceRef

 

Camel 2.9: Reference to lookup a timeoutCheckerExecutorService in the Registry

optimisticLocking

false

Camel 2.11: Turns on using optimistic locking, which requires the aggregationRepository being used, is supporting this by implementing the org.apache.camel.spi.OptimisticLockingAggregationRepository interface.

optimisticLockRetryPolicy

 

Camel 2.11.1: Allows to configure retry settings when using optimistic locking.

Exchange Properties

The following properties are set on each aggregated Exchange:

confluenceTableSmall

Header

Type

Description

CamelAggregatedSize

int

The total number of Exchanges aggregated into this combined Exchange.

CamelAggregatedCompletedBy

String

Indicator how the aggregation was completed as a value of either: predicate, size, strategy, consumer, timeout, forceCompletion or interval.

About AggregationStrategy

The AggregationStrategy is used for aggregating the old (lookup by its correlation id) and the new exchanges together into a single exchange. Possible implementations include performing some kind of combining or delta processing, such as adding line items together into an invoice or just using the newest exchange and removing old exchanges such as for state tracking or market data prices; where old values are of little use.

Notice the aggregation strategy is a mandatory option and must be provided to the aggregator.

Here are a few example AggregationStrategy implementations that should help you create your own custom strategy.

class ArrayListAggregationStrategy implements AggregationStrategy { public Exchange aggregate(Exchange oldExchange, Exchange newExchange) { Object newBody = newExchange.getIn().getBody(); ArrayList

Resequencer

The Resequencer from the EIP patterns allows you to reorganise messages based on some comparator. By default in Camel we use an Expression to create the comparator; so that you can compare by a message header or the body or a piece of a message etc.

Change in Camel 2.7

The <batch-config> and <stream-config> tags in XML DSL in the Resequencer EIP must now be configured in the top, and not in the bottom. So if you use those, then move them up just below the <resequence> EIP starts in the XML. If you are using Camel older than 2.7, then those configs should be at the bottom.

Camel supports two resequencing algorithms:

  • Batch resequencing collects messages into a batch, sorts the messages and sends them to their output.
  • Stream resequencing re-orders (continuous) message streams based on the detection of gaps between messages.

By default the Resequencer does not support duplicate messages and will only keep the last message, in case a message arrives with the same message expression. However in the batch mode you can enable it to allow duplicates.

Batch Resequencing

The following example shows how to use the batch-processing resequencer so that messages are sorted in order of the body() expression. That is messages are collected into a batch (either by a maximum number of messages per batch or using a timeout) then they are sorted in order and then sent out to their output.

Using the Fluent Builders{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ResequencerTest.java}This is equivalent to

from("direct:start") .resequence(body()).batch() .to("mock:result");

The batch-processing resequencer can be further configured via the size() and timeout() methods.

from("direct:start") .resequence(body()).batch().size(300).timeout(4000L) .to("mock:result")

This sets the batch size to 300 and the batch timeout to 4000 ms (by default, the batch size is 100 and the timeout is 1000 ms). Alternatively, you can provide a configuration object.

from("direct:start") .resequence(body()).batch(new BatchResequencerConfig(300, 4000L)) .to("mock:result")

So the above example will reorder messages from endpoint direct:a in order of their bodies, to the endpoint mock:result.
Typically you'd use a header rather than the body to order things; or maybe a part of the body. So you could replace this expression with

resequencer(header("mySeqNo"))

for example to reorder messages using a custom sequence number in the header mySeqNo.

You can of course use many different Expression languages such as XPath, XQuery, SQL or various Scripting Languages.

Using the Spring XML Extensions

xml<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start" /> <resequence> <simple>body</simple> <to uri="mock:result" /> <!-- batch-config can be ommitted for default (batch) resequencer settings --> <batch-config batchSize="300" batchTimeout="4000" /> </resequence> </route> </camelContext>

Allow Duplicates

Available as of Camel 2.4

In the batch mode, you can now allow duplicates. In Java DSL there is a allowDuplicates() method and in Spring XML there is an allowDuplicates=true attribute on the <batch-config/> you can use to enable it.

Reverse

Available as of Camel 2.4

In the batch mode, you can now reverse the expression ordering. By default the order is based on 0..9,A..Z, which would let messages with low numbers be ordered first, and thus also also outgoing first. In some cases you want to reverse order, which is now possible.

In Java DSL there is a reverse() method and in Spring XML there is an reverse=true attribute on the <batch-config/> you can use to enable it.

Resequence JMS messages based on JMSPriority

Available as of Camel 2.4

It's now much easier to use the Resequencer to resequence messages from JMS queues based on JMSPriority. For that to work you need to use the two new options allowDuplicates and reverse.{snippet:id=e1|lang=java|url=camel/trunk/components/camel-jms/src/test/java/org/apache/camel/component/jms/JmsBatchResequencerJMSPriorityTest.java}Notice this is only possible in the batch mode of the Resequencer.

Ignore invalid exchanges

Available as of Camel 2.9

The Resequencer EIP will from Camel 2.9 onwards throw a CamelExchangeException if the incoming Exchange is not valid for the resequencer - ie. the expression cannot be evaluated, such as a missing header. You can use the option ignoreInvalidExchanges to ignore these exceptions which means the Resequencer will then skip the invalid Exchange.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ResequenceBatchIgnoreInvalidExchangesTest.java}This option is available for both batch and stream resequencer.

Reject Old Exchanges

Available as of Camel 2.11

This option can be used to prevent out of order messages from being sent regardless of the event that delivered messages downstream (capacity, timeout, etc). If enabled using rejectOld(), the Resequencer will throw a MessageRejectedException when an incoming Exchange is "older" (based on the Comparator) than the last delivered message. This provides an extra level of control with regards to delayed message ordering.

from("direct:start") .onException(MessageRejectedException.class).handled(true).to("mock:error").end() .resequence(header("seqno")).stream().timeout(1000).rejectOld() .to("mock:result");

This option is available for the stream resequencer only.

Stream Resequencing

The next example shows how to use the stream-processing resequencer. Messages are re-ordered based on their sequence numbers given by a seqnum header using gap detection and timeouts on the level of individual messages.

Using the Fluent Builders{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/StreamResequencerTest.java}The stream-processing resequencer can be further configured via the capacity() and timeout() methods.

from("direct:start") .resequence(header("seqnum")).stream().capacity(5000).timeout(4000L) .to("mock:result")

This sets the resequencer's capacity to 5000 and the timeout to 4000 ms (by default, the capacity is 1000 and the timeout is 1000 ms). Alternatively, you can provide a configuration object.

from("direct:start") .resequence(header("seqnum")).stream(new StreamResequencerConfig(5000, 4000L)) .to("mock:result")

The stream-processing resequencer algorithm is based on the detection of gaps in a message stream rather than on a fixed batch size. Gap detection in combination with timeouts removes the constraint of having to know the number of messages of a sequence (i.e. the batch size) in advance. Messages must contain a unique sequence number for which a predecessor and a successor is known. For example a message with the sequence number 3 has a predecessor message with the sequence number 2 and a successor message with the sequence number 4. The message sequence 2,3,5 has a gap because the successor of 3 is missing. The resequencer therefore has to retain message 5 until message 4 arrives (or a timeout occurs).

If the maximum time difference between messages (with successor/predecessor relationship with respect to the sequence number) in a message stream is known, then the resequencer's timeout parameter should be set to this value. In this case it is guaranteed that all messages of a stream are delivered in correct order to the next processor. The lower the timeout value is compared to the out-of-sequence time difference the higher is the probability for out-of-sequence messages delivered by this resequencer. Large timeout values should be supported by sufficiently high capacity values. The capacity parameter is used to prevent the resequencer from running out of memory.

By default, the stream resequencer expects long sequence numbers but other sequence numbers types can be supported as well by providing a custom expression.{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/resequencer/MyFileNameExpression.java}{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/resequencer/ResequencerFileNameTest.java}or custom comparator via the comparator() method

ExpressionResultComparator<Exchange> comparator = new MyComparator(); from("direct:start") .resequence(header("seqnum")).stream().comparator(comparator) .to("mock:result");

or via a StreamResequencerConfig object.

ExpressionResultComparator<Exchange> comparator = new MyComparator(); StreamResequencerConfig config = new StreamResequencerConfig(100, 1000L, comparator); from("direct:start") .resequence(header("seqnum")).stream(config) .to("mock:result");

Using the Spring XML Extensions

xml<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <resequence> <simple>in.header.seqnum</simple> <to uri="mock:result" /> <stream-config capacity="5000" timeout="4000"/> </resequence> </route> </camelContext>

Further Examples

For further examples of this pattern in use you could look at the batch-processing resequencer junit test case and the stream-processing resequencer junit test case

Using This Pattern

Composed Message Processor

The Composed Message Processor from the EIP patterns allows you to process a composite message by splitting it up, routing the sub-messages to appropriate destinations and the re-aggregating the responses back into a single message.

In Camel we provide two solutions

The difference is when using only a Splitter it aggregates back all the splitted messages into the same aggregation group, eg like a fork/join pattern.
Whereas using the Aggregator allows you group into multiple groups, a pattern which provides more options.

Using the splitter alone is often easier and possibly a better solution. So take a look at this first, before involving the aggregator.

Example using both Splitter and Aggregator

In this example we want to check that a multipart order can be filled. Each part of the order requires a check at a different inventory.

Error formatting macro: snippet: java.lang.NullPointerException

Using the Spring XML Extensions

<route>
  <from uri="direct:start"/>
  <split>
    <simple>body</simple>
    <choice>
      <when>
        <method bean="orderItemHelper" method="isWidget"/>
	<to uri="bean:widgetInventory"/>
      </when>
      <otherwise>
	<to uri="bean:gadgetInventory"/>
      </otherwise>
    </choice>
    <to uri="seda:aggregate"/>
  </split>
</route>

<route>
  <from uri="seda:aggregate"/>
  <aggregate strategyRef="myOrderAggregatorStrategy" completionTimeout="1000">
    <correlationExpression>
      <simple>header.orderId</simple>
    </correlationExpression>
    <to uri="mock:result"/>
  </aggregate>
</route>

To do this we split up the order using a Splitter. The Splitter then sends individual OrderItems to a Content Based Router which checks the item type. Widget items get sent for checking in the widgetInventory bean and gadgets get sent to the gadgetInventory bean. Once these OrderItems have been validated by the appropriate bean, they are sent on to the Aggregator which collects and re-assembles the validated OrderItems into an order again.

When an order is sent it contains a header with the order id. We use this fact when we aggregate, as we configure this .header("orderId") on the aggregate DSL to instruct Camel to use the header with the key orderId as correlation expression.

For full details, check the example source here:

camel-core/src/test/java/org/apache/camel/processor/ComposedMessageProcessorTest.java

Example using only Splitter

In this example we want to split an incoming order using the Splitter eip, transform each order line, and then combine the order lines into a new order message.

Error formatting macro: snippet: java.lang.NullPointerException

Using XML

If you use XML, then the <split> tag offers the strategyRef attribute to refer to your custom AggregationStrategy

The bean with the methods to transform the order line and process the order as well:

Error formatting macro: snippet: java.lang.NullPointerException

And the AggregationStrategy we use with the Splitter eip to combine the orders back again (eg fork/join):

Error formatting macro: snippet: java.lang.NullPointerException

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Scatter-Gather

The Scatter-Gather from the EIP patterns allows you to route messages to a number of dynamically specified recipients and re-aggregate the responses back into a single message.

Dynamic Scatter-Gather Example

In this example we want to get the best quote for beer from several different vendors. We use a dynamic Recipient List to get the request for a quote to all vendors and an Aggregator to pick the best quote out of all the responses. The routes for this are defined as:

Error formatting macro: snippet: java.lang.NullPointerException

So in the first route you see that the Recipient List is looking at the listOfVendors header for the list of recipients. So, we need to send a message like

Error formatting macro: snippet: java.lang.NullPointerException

This message will be distributed to the following Endpoints: bean:vendor1, bean:vendor2, and bean:vendor3. These are all beans which look like

Error formatting macro: snippet: java.lang.NullPointerException

and are loaded up in Spring like

Error formatting macro: snippet: java.lang.NullPointerException

Each bean is loaded with a different price for beer. When the message is sent to each bean endpoint, it will arrive at the MyVendor.getQuote method. This method does a simple check whether this quote request is for beer and then sets the price of beer on the exchange for retrieval at a later step. The message is forwarded on to the next step using POJO Producing (see the @Produce annotation).

At the next step we want to take the beer quotes from all vendors and find out which one was the best (i.e. the lowest!). To do this we use an Aggregator with a custom aggregation strategy. The Aggregator needs to be able to compare only the messages from this particular quote; this is easily done by specifying a correlationExpression equal to the value of the quoteRequestId header. As shown above in the message sending snippet, we set this header to quoteRequest-1. This correlation value should be unique or you may include responses that are not part of this quote. To pick the lowest quote out of the set, we use a custom aggregation strategy like

Error formatting macro: snippet: java.lang.NullPointerException

Finally, we expect to get the lowest quote of $1 out of $1, $2, and $3.

Error formatting macro: snippet: java.lang.NullPointerException

You can find the full example source here:

camel-spring/src/test/java/org/apache/camel/spring/processor/scattergather/
camel-spring/src/test/resources/org/apache/camel/spring/processor/scattergather/scatter-gather.xml

Static Scatter-Gather Example

You can lock down which recipients are used in the Scatter-Gather by using a static Recipient List. It looks something like this

from("direct:start").multicast().to("seda:vendor1", "seda:vendor2", "seda:vendor3");

from("seda:vendor1").to("bean:vendor1").to("seda:quoteAggregator");
from("seda:vendor2").to("bean:vendor2").to("seda:quoteAggregator");
from("seda:vendor3").to("bean:vendor3").to("seda:quoteAggregator");

from("seda:quoteAggregator")
    .aggregate(header("quoteRequestId"), new LowestQuoteAggregationStrategy()).to("mock:result")

A full example of the static Scatter-Gather configuration can be found in the Loan Broker Example.

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Routing Slip

The Routing Slip from the EIP patterns allows you to route a message consecutively through a series of processing steps where the sequence of steps is not known at design time and can vary for each message.

Options

confluenceTableSmall

Name

Default Value

Description

uriDelimiter

,

Delimiter used if the Expression returned multiple endpoints.

ignoreInvalidEndpoints

false

If an endpoint URI could not be resolved, should it be ignored.

Otherwise, Camel will throw an exception stating the endpoint URI is not valid.

cacheSize

1000

Camel 2.13.1/2.12.4: Allows to configure the cache size for the ProducerCache which caches producers for reuse in the routing slip.

The default cache size is 1000.

A value of -1 disables the use of the cache.

Example

The following route will take any messages sent to the Apache ActiveMQ queue SomeQueue and pass them into the Routing Slip pattern.

javafrom("activemq:SomeQueue") .routingSlip("aRoutingSlipHeader");

Messages will be checked for the existence of the aRoutingSlipHeader header. The value of this header should be a comma-delimited list of endpoint URIs you wish the message to be routed to. The Message will be routed in a pipeline fashion, i.e., one after the other. From Camel 2.5 the Routing Slip will set a property, Exchange.SLIP_ENDPOINT, on the Exchange which contains the current endpoint as it advanced though the slip. This allows you to know how far we have processed in the slip.

The Routing Slip will compute the slip beforehand which means, the slip is only computed once. If you need to compute the slip on-the-fly then use the Dynamic Router pattern instead.

Configuration Options

Here we set the header name and the URI delimiter to something different.

Using the Fluent Builders{snippet:id=e3|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/routingslip/RoutingSlipTest.java}Using the Spring XML Extensions

<camelContext id="buildRoutingSlip" xmlns="http://activemq.apache.org/camel/schema/spring"> <route> <from uri="direct:c"/> <routingSlip uriDelimiter="#"> <header>aRoutingSlipHeader</header> </routingSlip> </route> </camelContext>

Ignore Invalid Endpoints

Available as of Camel 2.3

The Routing Slip now supports ignoreInvalidEndpoints which the Recipient List also supports. You can use it to skip endpoints which are invalid.

javafrom("direct:a") .routingSlip("myHeader") .ignoreInvalidEndpoints();

And in Spring XML its an attribute on the recipient list tag:

xml<route> <from uri="direct:a"/> <routingSlip ignoreInvalidEndpoints="true"/> <header>myHeader</header> </routingSlip> </route>

Then let's say the myHeader contains the following two endpoints direct:foo,xxx:bar. The first endpoint is valid and works. However the second endpoint is invalid and will just be ignored. Camel logs at INFO level, so you can see why the endpoint was invalid.

Expression Support

Available as of Camel 2.4

The Routing Slip now supports to take the expression parameter as the Recipient List does. You can tell Camel the expression that you want to use to get the routing slip.

javafrom("direct:a") .routingSlip(header("myHeader")) .ignoreInvalidEndpoints();

And in Spring XML its an attribute on the recipient list tag.

xml<route> <from uri="direct:a"/> <!--NOTE from Camel 2.4.0, you need to specify the expression element inside of the routingSlip element --> <routingSlip ignoreInvalidEndpoints="true"> <header>myHeader</header> </routingSlip> </route>

Further Examples

For further examples of this pattern in use you could look at the routing slip test cases.

Using This Pattern

Throttler

The Throttler Pattern allows you to ensure that a specific endpoint does not get overloaded, or that we don't exceed an agreed SLA with some external service.

Options

confluenceTableSmall

Name

Default Value

Description

maximumRequestsPerPeriod

 

Maximum number of requests per period to throttle. This option must be provided as a positive number. Notice, in the XML DSL, from Camel 2.8 onwards this option is configured using an Expression instead of an attribute.

timePeriodMillis

1000

The time period in milliseconds, in which the throttler will allow at most maximumRequestsPerPeriod number of messages.

asyncDelayed

false

Camel 2.4: If enabled then any messages which is delayed happens asynchronously using a scheduled thread pool.

executorServiceRef

 

Camel 2.4: Refers to a custom Thread Pool to be used if asyncDelay has been enabled.

callerRunsWhenRejected

true

Camel 2.4: Is used if asyncDelayed was enabled. This controls if the caller thread should execute the task if the thread pool rejected the task.

rejectExecution

false

Camel 2.14: If this option is true, throttler throws a ThrottlerRejectExecutionException when the request rate exceeds the limit.

Examples

Using the Fluent Builders

{snippet:id=ex|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ThrottlerTest.java}

So the above example will throttle messages all messages received on seda:a before being sent to mock:result ensuring that a maximum of 3 messages are sent in any 10 second window.

Note that since timePeriodMillis defaults to 1000 milliseconds, just setting the maximumRequestsPerPeriod has the effect of setting the maximum number of requests per second. So to throttle requests at 100 requests per second between two endpoints, it would look more like this...

from("seda:a").throttle(100).to("seda:b");

For further examples of this pattern in use you could look at the junit test case

Using the Spring XML Extensions

Camel 2.7.x or older

{snippet:id=example|lang=xml|url=camel/tags/camel-2.7.0/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/throttler.xml}

Camel 2.8 onwards

In Camel 2.8 onwards you must set the maximum period as an Expression as shown below where we use a Constant expression:

{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/throttler.xml}

Dynamically changing maximum requests per period

Available as of Camel 2.8
Since we use an Expression you can adjust this value at runtime, for example you can provide a header with the value. At runtime Camel evaluates the expression and converts the result to a java.lang.Long type. In the example below we use a header from the message to determine the maximum requests per period. If the header is absent, then the Throttler uses the old value. So that allows you to only provide a header if the value is to be changed:

{snippet:id=e2|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/throttler.xml}

Asynchronous delaying

Available as of Camel 2.4

You can let the Throttler use non blocking asynchronous delaying, which means Camel will use a scheduler to schedule a task to be executed in the future. The task will then continue routing. This allows the caller thread to not block and be able to service other messages, etc.

from("seda:a").throttle(100).asyncDelayed().to("seda:b");

Using This Pattern

Sampling Throttler

Available as of Camel 2.1

A sampling throttler allows you to extract a sample of the exchanges from the traffic through a route.
It is configured with a sampling period during which only a single exchange is allowed to pass through. All other exchanges will be stopped.

Will by default use a sample period of 1 seconds.

Options

confluenceTableSmall

Name

Default Value

Description

messageFrequency

 

Samples the message every N'th message. You can only use either frequency or period.

samplePeriod

1

Samples the message every N'th period. You can only use either frequency or period.

units

SECOND

Time unit as an enum of java.util.concurrent.TimeUnit from the JDK.

Samples

You use this EIP with the sample DSL as show in these samples.

Using the Fluent Builders
These samples also show how you can use the different syntax to configure the sampling period:

{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/SamplingThrottlerTest.java}

Using the Spring XML Extensions
And the same example in Spring XML is:

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/samplingThrottler.xml}

And since it uses a default of 1 second you can omit this configuration in case you also want to use 1 second

{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/samplingThrottlerWithDefault.xml} Using This Pattern

See Also

Delayer

The Delayer Pattern allows you to delay the delivery of messages to some destination.

Delayer

The expression is a value in millis to wait from the current time, so the expression should just be 3000.
However you can use a long value for a fixed value to indicate the delay in millis.
See the Spring DSL samples for Delayer.

Using Delayer in Java DSL

See this ticket: https://issues.apache.org/jira/browse/CAMEL-2654

Options

confluenceTableSmall

Name

Default Value

Description

asyncDelayed

false

Camel 2.4: If enabled then delayed messages happens asynchronously using a scheduled thread pool.

executorServiceRef

 

Camel 2.4: Refers to a custom Thread Pool to be used if asyncDelay has been enabled.

callerRunsWhenRejected

true

Camel 2.4: Is used if asyncDelayed was enabled. This controls if the caller thread should execute the task if the thread pool rejected the task.

Using the Fluent Builders

The example below will delay all messages received on seda:b 1 second before sending them to mock:result.

{snippet:id=ex2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/DelayerTest.java}

You can just delay things a fixed amount of time from the point at which the delayer receives the message. For example to delay things 2 seconds

delayer(2000)

The above assume that the delivery order is maintained and that the messages are delivered in delay order. If you want to reorder the messages based on delivery time, you can use the Resequencer with this pattern. For example

from("activemq:someQueue").resequencer(header("MyDeliveryTime")).delay("MyRedeliveryTime").to("activemq:aDelayedQueue");

You can of course use many different Expression languages such as XPath, XQuery, SQL or various Scripting Languages. For example to delay the message for the time period specified in the header, use the following syntax:

from("activemq:someQueue").delay(header("delayValue")).to("activemq:aDelayedQueue");

And to delay processing using the Simple language you can use the following DSL:

from("activemq:someQueue").delay(simple("${body.delayProperty}")).to("activemq:aDelayedQueue");

Spring DSL

The sample below demonstrates the delay in Spring DSL:

{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/delayer.xml}

For further examples of this pattern in use you could look at the junit test case

Asynchronous delaying

Available as of Camel 2.4

You can let the Delayer use non blocking asynchronous delaying, which means Camel will use a scheduler to schedule a task to be executed in the future. The task will then continue routing. This allows the caller thread to not block and be able to service other messages etc.

From Java DSL

You use the asyncDelayed() to enable the async behavior.

from("activemq:queue:foo").delay(1000).asyncDelayed().to("activemq:aDelayedQueue");

From Spring XML

You use the asyncDelayed="true" attribute to enable the async behavior.

xml<route> <from uri="activemq:queue:foo"/> <delay asyncDelayed="true"> <constant>1000</constant> </delay> <to uri="activemq:aDealyedQueue"/> </route>

Creating a custom delay

You can use an expression to determine when to send a message using something like this

from("activemq:foo"). delay().method("someBean", "computeDelay"). to("activemq:bar");

then the bean would look like this...

public class SomeBean { public long computeDelay() { long delay = 0; // use java code to compute a delay value in millis return delay; } }

Using This Pattern

See Also

Load Balancer

The Load Balancer Pattern allows you to delegate to one of a number of endpoints using a variety of different load balancing policies.

Built-in load balancing policies

Camel provides the following policies out-of-the-box:

Policy

Description

Round Robin

The exchanges are selected from in a round robin fashion. This is a well known and classic policy, which spreads the load evenly.

Random

A random endpoint is selected for each exchange.

Sticky

Sticky load balancing using an Expression to calculate a correlation key to perform the sticky load balancing; rather like jsessionid in the web or JMSXGroupID in JMS.

Topic

Topic which sends to all destinations (rather like JMS Topics)

Failover

In case of failures the exchange will be tried on the next endpoint.

Weighted Round-Robin

Camel 2.5: The weighted load balancing policy allows you to specify a processing load distribution ratio for each server with respect to the others. In addition to the weight, endpoint selection is then further refined using round-robin distribution based on weight.

Weighted Random

Camel 2.5: The weighted load balancing policy allows you to specify a processing load distribution ratio for each server with respect to others.In addition to the weight, endpoint selection is then further refined using random distribution based on weight.

Custom

Camel 2.8: From Camel 2.8 onwards the preferred way of using a custom Load Balancer is to use this policy, instead of using the @deprecated ref attribute.

Circuit Breaker

Camel 2.14: Implements the Circuit Breaker pattern as described in "Release it!" book.

Load balancing HTTP endpoints

If you are proxying and load balancing HTTP, then see this page for more details.

Round Robin

The round robin load balancer is not meant to work with failover, for that you should use the dedicated failover load balancer. The round robin load balancer will only change to next endpoint per message.

The round robin load balancer is stateful as it keeps state of which endpoint to use next time.

Using the Fluent Builders{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RoundRobinLoadBalanceTest.java}Using the Spring configuration

xml<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <loadBalance> <roundRobin/> <to uri="mock:x"/> <to uri="mock:y"/> <to uri="mock:z"/> </loadBalance> </route> </camelContext>

The above example loads balance requests from direct:start to one of the available mock endpoint instances, in this case using a round robin policy.
For further examples of this pattern look at this junit test case

Failover

The failover load balancer is capable of trying the next processor in case an Exchange failed with an exception during processing.
You can constrain the failover to activate only when one exception of a list you specify occurs. If you do not specify a list any exception will cause fail over to occur. This balancer uses the same strategy for matching exceptions as the Exception Clause does for the onException.

Enable stream caching if using streams

If you use streaming then you should enable Stream caching when using the failover load balancer. This is needed so the stream can be re-read after failing over to the next processor.

Failover offers the following options:

Option

Type

Default

Description

inheritErrorHandler

boolean

true

Camel 2.3: Whether or not the Error Handler configured on the route should be used. Disable this if you want failover to transfer immediately to the next endpoint. On the other hand, if you have this option enabled, then Camel will first let the Error Handler try to process the message. The Error Handler may have been configured to redeliver and use delays between attempts. If you have enabled a number of redeliveries then Camel will try to redeliver to the same endpoint, and only fail over to the next endpoint, when the Error Handler is exhausted.

maximumFailoverAttempts

int

-1

Camel 2.3: A value to indicate after X failover attempts we should exhaust (give up). Use -1 to indicate never give up and continuously try to failover. Use 0 to never failover. And use e.g. 3 to failover at most 3 times before giving up. This option can be used whether or not roundRobin is enabled or not.

roundRobin

boolean

false

Camel 2.3: Whether or not the failover load balancer should operate in round robin mode or not. If not, then it will always start from the first endpoint when a new message is to be processed. In other words it restart from the top for every message. If round robin is enabled, then it keeps state and will continue with the next endpoint in a round robin fashion. When using round robin it will not stick to last known good endpoint, it will always pick the next endpoint to use. You can also enable sticky mode together with round robin, if so then it will pick the last known good endpoint to use when starting the load balancing (instead of using the next when starting).

stickybooleanfalseCamel 2.16: Whether or not the failover load balancer should operate in sticky mode or not. If not, then it will always start from the first endpoint when a new message is to be processed. In other words it restart from the top for every message. If sticky is enabled, then it keeps state and will continue with the last known good endpoint. You can also enable sticky mode together with round robin, if so then it will pick the last known good endpoint to use when starting the load balancing (instead of using the next when starting).

Camel 2.2 or older behavior
The current implementation of failover load balancer uses simple logic which always tries the first endpoint, and in case of an exception being thrown it tries the next in the list, and so forth. It has no state, and the next message will thus always start with the first endpoint.

Camel 2.3 onwards behavior
The failover load balancer now supports round robin mode, which allows you to failover in a round robin fashion. See the roundRobin option.

Redelivery must be enabled

In Camel 2.2 or older the failover load balancer requires you have enabled Camel Error Handler to use redelivery. In Camel 2.3 onwards this is not required as such, as you can mix and match. See the inheritErrorHandler option.

Here is a sample to failover only if a IOException related exception was thrown:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/FailOverNotCatchedExceptionTest.java}You can specify multiple exceptions to failover as the option is varargs, for instance:

java// enable redelivery so failover can react errorHandler(defaultErrorHandler().maximumRedeliveries(5)); from("direct:foo"). loadBalance().failover(IOException.class, MyOtherException.class) .to("direct:a", "direct:b");

Using failover in Spring DSL

Failover can also be used from Spring DSL and you configure it as:

xml <route errorHandlerRef="myErrorHandler"> <from uri="direct:foo"/> <loadBalance> <failover> <exception>java.io.IOException</exception> <exception>com.mycompany.MyOtherException</exception> </failover> <to uri="direct:a"/> <to uri="direct:b"/> </loadBalance> </route>

Using failover in round robin mode

An example using Java DSL:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/FailoverRoundRobinTest.java}And the same example using Spring XML:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/FailoverRoundRobinTest.xml}

Disabled inheritErrorHandler

You can configure inheritErrorHandler=false if you want to failover to the next endpoint as fast as possible. By disabling the Error Handler you ensure it does not intervene which allows the failover load balancer to handle failover asap. By also enabling roundRobin mode, then it will keep retrying until it success. You can then configure the maximumFailoverAttempts option to a high value to let it eventually exhaust (give up) and fail.

Weighted Round-Robin and Random Load Balancing

Available as of Camel 2.5

In many enterprise environments where server nodes of unequal processing power & performance characteristics are utilized to host services and processing endpoints, it is frequently necessary to distribute processing load based on their individual server capabilities so that some endpoints are not unfairly burdened with requests. Obviously simple round-robin or random load balancing do not alleviate problems of this nature. A Weighted Round-Robin and/or Weighted Random load balancer can be used to address this problem.

The weighted load balancing policy allows you to specify a processing load distribution ratio for each server with respect to others. You can specify this as a positive processing weight for each server. A larger number indicates that the server can handle a larger load. The weight is utilized to determine the payload distribution ratio to different processing endpoints with respect to others.

Disabled inheritErrorHandler

As of Camel 2.6, the Weighted Load balancer usage has been further simplified, there is no need to send in distributionRatio as a List<Integer>. It can be simply sent as a delimited String of integer weights separated by a delimiter of choice.

The parameters that can be used are

In Camel 2.5

Option

Type

Default

Description

roundRobin

boolean

false

The default value for round-robin is false. In the absence of this setting or parameter the load balancing algorithm used is random.

distributionRatio

List<Integer>

none

The distributionRatio is a list consisting on integer weights passed in as a parameter. The distributionRatio must match the number of endpoints and/or processors specified in the load balancer list. In Camel 2.5 if endpoints do not match ratios, then a best effort distribution is attempted.

Available In Camel 2.6

Option

Type

Default

Description

roundRobin

boolean

false

The default value for round-robin is false. In the absence of this setting or parameter the load balancing algorithm used is random.

distributionRatio

String

none

The distributionRatio is a delimited String consisting on integer weights separated by delimiters for example "2,3,5". The distributionRatio must match the number of endpoints and/or processors specified in the load balancer list.

distributionRatioDelimiter

String

,

The distributionRatioDelimiter is the delimiter used to specify the distributionRatio. If this attribute is not specified a default delimiter "," is expected as the delimiter used for specifying the distributionRatio.

Using Weighted round-robin & random load balancing

In Camel 2.5

An example using Java DSL:

javaArrayList<integer> distributionRatio = new ArrayList<integer>(); distributionRatio.add(4); distributionRatio.add(2); distributionRatio.add(1); // round-robin from("direct:start") .loadBalance().weighted(true, distributionRatio) .to("mock:x", "mock:y", "mock:z"); //random from("direct:start") .loadBalance().weighted(false, distributionRatio) .to("mock:x", "mock:y", "mock:z");

And the same example using Spring XML:

xml <route> <from uri="direct:start"/> <loadBalance> <weighted roundRobin="false" distributionRatio="4 2 1"/> <to uri="mock:x"/> <to uri="mock:y"/> <to uri="mock:z"/> </loadBalance> </route>

Available In Camel 2.6

An example using Java DSL:

java// round-robin from("direct:start") .loadBalance().weighted(true, "4:2:1" distributionRatioDelimiter=":") .to("mock:x", "mock:y", "mock:z"); //random from("direct:start") .loadBalance().weighted(false, "4,2,1") .to("mock:x", "mock:y", "mock:z");

And the same example using Spring XML:

xml <route> <from uri="direct:start"/> <loadBalance> <weighted roundRobin="false" distributionRatio="4-2-1" distributionRatioDelimiter="-" /> <to uri="mock:x"/> <to uri="mock:y"/> <to uri="mock:z"/> </loadBalance> </route>

Custom Load Balancer

You can use a custom load balancer (eg your own implementation) also.

An example using Java DSL:{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/CustomLoadBalanceTest.java}And the same example using XML DSL:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringCustomRefLoadBalanceTest.xml}Notice in the XML DSL above we use <custom> which is only available in Camel 2.8 onwards. In older releases you would have to do as follows instead:

xml <loadBalance ref="myBalancer"> <!-- these are the endpoints to balancer --> <to uri="mock:x"/> <to uri="mock:y"/> <to uri="mock:z"/> </loadBalance>

To implement a custom load balancer you can extend some support classes such as LoadBalancerSupport and SimpleLoadBalancerSupport. The former supports the asynchronous routing engine, and the latter does not. Here is an example:{snippet:id=e2|title=Custom load balancer implementation|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/CustomLoadBalanceTest.java}

Circuit Breaker

The Circuit Breaker load balancer is a stateful pattern that monitors all calls for certain exceptions. Initially the Circuit Breaker is in closed state and passes all messages. If there are failures and the threshold is reached, it moves to open state and rejects all calls until halfOpenAfter timeout is reached. After this timeout is reached, if there is a new call, it will pass and if the result is success the Circuit Breaker will move to closed state, or to open state if there was an error.

When the circuit breaker is closed, it will throw a java.util.concurrent.RejectedExecutionException. This can then be caught to provide an alternate path for processing exchanges.

An example using Java DSL:

javafrom("direct:start") .onException(RejectedExecutionException.class) .handled(true) .to("mock:serviceUnavailable") .end()  .loadBalance() .circuitBreaker(2, 1000L, MyCustomException.class) .to("mock:service") .end();

And the same example using Spring XML:

xml<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <onException> <exception>java.util.concurrent.RejectedExecutionException</exception> <handled><constant>true</constant></handled> <to uri="mock:serviceUnavailable"/> </onException>  <loadBalance> <circuitBreaker threshold="2" halfOpenAfter="1000"> <exception>MyCustomException</exception> </circuitBreaker> <to uri="mock:service"/> </loadBalance> </route> </camelContext>

Using This Pattern

Multicast

The Multicast allows to route the same message to a number of endpoints and process them in a different way. The main difference between the Multicast and Splitter is that Splitter will split the message into several pieces but the Multicast will not modify the request message.

Options

confluenceTableSmall

Name

Default Value

Description

strategyRef

 

Refers to an AggregationStrategy to be used to assemble the replies from the multicasts, into a single outgoing message from the Multicast. By default Camel will use the last reply as the outgoing message. From Camel 2.12 onwards you can also use a POJO as the AggregationStrategy, see the Aggregate page for more details. If an exception is thrown from the aggregate method in the AggregationStrategy, then by default, that exception is not handled by the error handler. The error handler can be enabled to react if enabling the shareUnitOfWork option.

strategyMethodName

 

Camel 2.12: This option can be used to explicit declare the method name to use, when using POJOs as the AggregationStrategy. See the Aggregate page for more details.

strategyMethodAllowNull

false

Camel 2.12: If this option is false then the aggregate method is not used if there was no data to enrich. If this option is true then null values is used as the oldExchange (when no data to enrich), when using POJOs as the AggregationStrategy. See the Aggregate page for more details.

parallelProcessing

false

If enabled then sending messages to the multicasts occurs concurrently. Note the caller thread will still wait until all messages has been fully processed, before it continues. Its only the sending and processing the replies from the multicasts which happens concurrently.

 

parallelAggregate

false

Camel 2.14: If enabled then the aggregate method on AggregationStrategy can be called concurrently. Notice that this would require the implementation of AggregationStrategy to be implemented as thread-safe. By default this is false meaning that Camel synchronizes the call to the aggregate method. Though in some use-cases this can be used to archive higher performance when the AggregationStrategy is implemented as thread-safe.

executorServiceRef

 

Refers to a custom Thread Pool to be used for parallel processing. Notice if you set this option, then parallel processing is automatic implied, and you do not have to enable that option as well.

stopOnException

false

Camel 2.2: Whether or not to stop continue processing immediately when an exception occurred. If disable, then Camel will send the message to all multicasts regardless if one of them failed. You can deal with exceptions in the AggregationStrategy class where you have full control how to handle that.

streaming

false

If enabled then Camel will process replies out-of-order, eg in the order they come back. If disabled, Camel will process replies in the same order as multicasted.

timeout

 

Camel 2.5: Sets a total timeout specified in millis. If the Multicast hasn't been able to send and process all replies within the given timeframe, then the timeout triggers and the Multicast breaks out and continues. Notice if you provide a TimeoutAwareAggregationStrategy then the timeout method is invoked before breaking out. If the timeout is reached with running tasks still remaining, certain tasks for which it is difficult for Camel to shut down in a graceful manner may continue to run. So use this option with a bit of care. We may be able to improve this functionality in future Camel releases.

onPrepareRef

 

Camel 2.8: Refers to a custom Processor to prepare the copy of the Exchange each multicast will receive. This allows you to do any custom logic, such as deep-cloning the message payload if that's needed etc.

shareUnitOfWork

false

Camel 2.8: Whether the unit of work should be shared. See the same option on Splitter for more details.

Example

The following example shows how to take a request from the direct:a endpoint , then multicast these request to direct:x, direct:y, direct:z.

Using the Fluent Builders{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/MulticastTest.java}By default Multicast invokes each endpoint sequentially. If parallel processing is desired, simply use

from("direct:a").multicast().parallelProcessing().to("direct:x", "direct:y", "direct:z");

In case of using InOut MEP, an AggregationStrategy is used for aggregating all reply messages. The default is to only use the latest reply message and discard any earlier replies. The aggregation strategy is configurable:

from("direct:start") .multicast(new MyAggregationStrategy()) .parallelProcessing().timeout(500).to("direct:a", "direct:b", "direct:c") .end() .to("mock:result");

Stop processing in case of exception

Available as of Camel 2.1

The Multicast will by default continue to process the entire Exchange even in case one of the multicasted messages will thrown an exception during routing.
For example if you want to multicast to 3 destinations and the 2nd destination fails by an exception. What Camel does by default is to process the remainder destinations. You have the chance to remedy or handle this in the AggregationStrategy.

But sometimes you just want Camel to stop and let the exception be propagated back, and let the Camel error handler handle it. You can do this in Camel 2.1 by specifying that it should stop in case of an exception occurred. This is done by the stopOnException option as shown below:

from("direct:start") .multicast() .stopOnException().to("direct:foo", "direct:bar", "direct:baz") .end() .to("mock:result"); from("direct:foo").to("mock:foo"); from("direct:bar").process(new MyProcessor()).to("mock:bar"); from("direct:baz").to("mock:baz");

And using XML DSL you specify it as follows:

xml <route> <from uri="direct:start"/> <multicast stopOnException="true"> <to uri="direct:foo"/> <to uri="direct:bar"/> <to uri="direct:baz"/> </multicast> <to uri="mock:result"/> </route> <route> <from uri="direct:foo"/> <to uri="mock:foo"/> </route> <route> <from uri="direct:bar"/> <process ref="myProcessor"/> <to uri="mock:bar"/> </route> <route> <from uri="direct:baz"/> <to uri="mock:baz"/> </route>

Using onPrepare to execute custom logic when preparing messages

Available as of Camel 2.8

The Multicast will copy the source Exchange and multicast each copy. However the copy is a shallow copy, so in case you have mutateable message bodies, then any changes will be visible by the other copied messages. If you want to use a deep clone copy then you need to use a custom onPrepare which allows you to do this using the Processor interface.

Notice the onPrepare can be used for any kind of custom logic which you would like to execute before the Exchange is being multicasted.

Design for immutable

Its best practice to design for immutable objects.

For example if you have a mutable message body as this Animal class:{snippet:id=e1|lang=java|title=Animal|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/Animal.java}Then we can create a deep clone processor which clones the message body:{snippet:id=e1|lang=java|title=AnimalDeepClonePrepare|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/AnimalDeepClonePrepare.java}Then we can use the AnimalDeepClonePrepare class in the Multicast route using the onPrepare option as shown:{snippet:id=e1|lang=java|title=Multicast using onPrepare|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/MulticastOnPrepareTest.java}And the same example in XML DSL{snippet:id=e1|lang=xml|title=Multicast using onPrepare|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/MulticastOnPrepareTest.xml}Notice the onPrepare option is also available on other EIPs such as Splitter, Recipient List, and Wire Tap.

Using This Pattern

Loop

The Loop allows for processing a message a number of times, possibly in a different way for each iteration. Useful mostly during testing.

Default mode

Notice by default the loop uses the same exchange throughout the looping. So the result from the previous iteration will be used for the next (eg Pipes and Filters). From Camel 2.8 onwards you can enable copy mode instead. See the options table for more details.

Options

confluenceTableSmall 

Name

Default Value

Description

copy

false

Camel 2.8: Whether or not copy mode is used. If false then the same Exchange will be used for each iteration. So the result from the previous iteration will be visible for the next iteration. Instead you can enable copy mode, and then each iteration restarts with a fresh copy of the input Exchange.

doWhile Camel 2.17: Enables the while loop that loops until the predicate evaluates to false or null.

Exchange properties

For each iteration two properties are set on the Exchange. Processors can rely on these properties to process the Message in different ways.

Property

Description

CamelLoopSize

Total number of loops. This is not available if running the loop in while loop mode.

CamelLoopIndex

Index of the current iteration (0 based)

Examples

The following example shows how to take a request from the direct:x endpoint, then send the message repetitively to mock:result. The number of times the message is sent is either passed as an argument to loop(), or determined at runtime by evaluating an expression. The expression must evaluate to an int, otherwise a RuntimeCamelException is thrown.

Using the Fluent Builders

Pass loop count as an argument{snippet:id=ex1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/LoopTest.java}Use expression to determine loop count{snippet:id=ex2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/LoopTest.java}Use expression to determine loop count{snippet:id=ex3|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/LoopTest.java}Using the Spring XML Extensions

Pass loop count as an argument{snippet:id=ex1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/loop.xml}Use expression to determine loop count{snippet:id=ex2|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/loop.xml}For further examples of this pattern in use you could look at one of the junit test case

Using copy mode

Available as of Camel 2.8

Now suppose we send a message to "direct:start" endpoint containing the letter A.
The output of processing this route will be that, each "mock:loop" endpoint will receive "AB" as message.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/LoopCopyTest.java}However if we do not enable copy mode then "mock:loop" will receive "AB", "ABB", "ABBB", etc. messages.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/LoopNoCopyTest.java}The equivalent example in XML DSL in copy mode is as follows:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringLoopCopyTest.xml}

Using while mode

Available as of Camel 2.17

The loop can act like a while loop that loops until the expression evaluates to false or null.

For example the route below loops while the length of the message body is 5 or less characters. Notice that the DSL uses loopDoWhile.

from("direct:start") .loopDoWhile(simple("${body.length} <= 5")) .to("mock:loop") .transform(body().append("A")) .end() .to("mock:result");

And the same example in XML:

xml <route> <from uri="direct:start"/> <loop doWhile="true"> <simple>${body.length} &lt;= 5</simple> <to uri="mock:loop"/> <transform> <simple>A${body}</simple> </transform> </loop> <to uri="mock:result"/> </route>

Notice in XML that the while loop is turned on using the doWhile attribute.

 

Using This Pattern

Message Transformation

Error rendering macro 'include'

null

Content Filter

Camel supports the Content Filter from the EIP patterns using one of the following mechanisms in the routing logic to transform content from the inbound message.

A common way to filter messages is to use an Expression in the DSL like XQuery, SQL or one of the supported Scripting Languages.

Using the Fluent Builders

Here is a simple example using the DSL directly

Error formatting macro: snippet: java.lang.NullPointerException

In this example we add our own Processor

Error formatting macro: snippet: java.lang.NullPointerException

For further examples of this pattern in use you could look at one of the JUnit tests

Using Spring XML

<route>
  <from uri="activemq:Input"/>
  <bean ref="myBeanName" method="doTransform"/>
  <to uri="activemq:Output"/>
</route>

You can also use XPath to filter out part of the message you are interested in:

<route>
  <from uri="activemq:Input"/>
  <setBody><xpath resultType="org.w3c.dom.Document">//foo:bar</xpath></setBody>
  <to uri="activemq:Output"/>
</route> 

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Claim Check

The Claim Check from the EIP patterns allows you to replace message content with a claim check (a unique key), which can be used to retrieve the message content at a later time. The message content is stored temporarily in a persistent store like a database or file system. This pattern is very useful when message content is very large (thus it would be expensive to send around) and not all components require all information.

It can also be useful in situations where you cannot trust the information with an outside party; in this case, you can use the Claim Check to hide the sensitive portions of data.

Example

In this example we want to replace a message body with a claim check, and restore the body at a later step.

Using the Fluent Builders

{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ClaimCheckTest.java}

Using the Spring XML Extensions

xml <route> <from uri="direct:start"/> <pipeline> <to uri="bean:checkLuggage"/> <to uri="mock:testCheckpoint"/> <to uri="bean:dataEnricher"/> <to uri="mock:result"/> </pipeline> </route>

The example route is pretty simple - its just a Pipeline. In a real application you would have some other steps where the mock:testCheckpoint endpoint is in the example.

The message is first sent to the checkLuggage bean which looks like

{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ClaimCheckTest.java}

This bean stores the message body into the data store, using the custId as the claim check. In this example, we're just using a HashMap to store the message body; in a real application you would use a database or file system, etc. Next the claim check is added as a message header for use later. Finally we remove the body from the message and pass it down the pipeline.

The next step in the pipeline is the mock:testCheckpoint endpoint which is just used to check that the message body is removed, claim check added, etc.

To add the message body back into the message, we use the dataEnricher bean which looks like

{snippet:id=e3|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ClaimCheckTest.java}

This bean queries the data store using the claim check as the key and then adds the data back into the message. The message body is then removed from the data store and finally the claim check is removed. Now the message is back to what we started with!

For full details, check the example source here:

camel-core/src/test/java/org/apache/camel/processor/ClaimCheckTest.java

Using This Pattern

Normalizer

Camel supports the Normalizer from the EIP patterns by using a Message Router in front of a number of Message Translator instances.

Example

This example shows a Message Normalizer that converts two types of XML messages into a common format. Messages in this common format are then filtered.

Using the Fluent Builders

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/NormalizerTest.java}

In this case we're using a Java bean as the normalizer. The class looks like this

{snippet:id=example|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/MyNormalizer.java}

Using the Spring XML Extensions

The same example in the Spring DSL

{snippet:id=example|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/normalizer.xml}

See Also

Using This Pattern

Sort

Sort can be used to sort a message. Imagine you consume text files and before processing each file you want to be sure the content is sorted.

Sort will by default sort the body using a default comparator that handles numeric values or uses the string representation. You can provide your own comparator, and even an expression to return the value to be sorted. Sort requires the value returned from the expression evaluation is convertible to java.util.List as this is required by the JDK sort operation.

Options

Name

Default Value

Description

comparatorRef

 

Refers to a custom java.util.Comparator to use for sorting the message body. Camel will by default use a comparator which does a A..Z sorting.

Using from Java DSL

In the route below it will read the file content and tokenize by line breaks so each line can be sorted.

from("file://inbox").sort(body().tokenize("\n")).to("bean:MyServiceBean.processLine");

You can pass in your own comparator as a 2nd argument:

from("file://inbox").sort(body().tokenize("\n"), new MyReverseComparator()).to("bean:MyServiceBean.processLine");

Using from Spring DSL

In the route below it will read the file content and tokenize by line breaks so each line can be sorted.

Camel 2.7 or better
<route>
  <from uri="file://inbox"/>
  <sort>
    <simple>body</simple>
  </sort>
  <beanRef ref="myServiceBean" method="processLine"/>
</route>
Camel 2.6 or older
<route>
  <from uri="file://inbox"/>
  <sort>
    <expression>
      <simple>body</simple>
    </expression>
  </sort>
  <beanRef ref="myServiceBean" method="processLine"/>
</route>

And to use our own comparator we can refer to it as a spring bean:

Camel 2.7 or better
<route>
  <from uri="file://inbox"/>
  <sort comparatorRef="myReverseComparator">
    <simple>body</simple>
  </sort>
  <beanRef ref="MyServiceBean" method="processLine"/>
</route>

<bean id="myReverseComparator" class="com.mycompany.MyReverseComparator"/>
Camel 2.6 or older
<route>
  <from uri="file://inbox"/>
  <sort comparatorRef="myReverseComparator">
    <expression>
      <simple>body</simple>
    </expression>
  </sort>
  <beanRef ref="MyServiceBean" method="processLine"/>
</route>

<bean id="myReverseComparator" class="com.mycompany.MyReverseComparator"/>

Besides <simple>, you can supply an expression using any language you like, so long as it returns a list.

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Messaging Endpoints

Messaging Mapper

Camel supports the Messaging Mapper from the EIP patterns by using either Message Translator pattern or the Type Converter module.

Example

The following example demonstrates the use of a Bean component to map between two messaging system

Using the Fluent Builders

from("activemq:foo")
	.beanRef("transformerBean", "transform")
	.to("jms:bar");

 

Using the Spring XML Extensions

<route>
	<from uri="activemq:foo"/>
	<bean ref="transformerBean" method="transform" />
	<to uri="jms:bar"/>
</route>

See also

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Event Driven Consumer

Camel supports the Event Driven Consumer from the EIP patterns. The default consumer model is event based (i.e. asynchronous) as this means that the Camel container can then manage pooling, threading and concurrency for you in a declarative manner.

The Event Driven Consumer is implemented by consumers implementing the Processor interface which is invoked by the Message Endpoint when a Message is available for processing.

Example

The following demonstrates a Processor defined in the Camel  Registry which is invoked when an event occurs from a JMS queue


Using the Fluent Builders

from("jms:queue:foo")
	.processRef("processor");

 

Using the Spring XML Extensions

<route>
	<from uri="jms:queue:foo"/>
	<to uri="processor"/>
</route>

 

For more details see

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Polling Consumer

Camel supports implementing the Polling Consumer from the EIP patterns using the PollingConsumer interface which can be created via the Endpoint.createPollingConsumer() method.

In Java:

javaEndpoint endpoint = context.getEndpoint("activemq:my.queue"); PollingConsumer consumer = endpoint.createPollingConsumer(); Exchange exchange = consumer.receive();

The ConsumerTemplate (discussed below) is also available.

There are three main polling methods on PollingConsumer

Method name

Description

receive()

Waits until a message is available and then returns it; potentially blocking forever

receive(long)

Attempts to receive a message exchange, waiting up to the given timeout and returning null if no message exchange could be received within the time available

receiveNoWait()

Attempts to receive a message exchange immediately without waiting and returning null if a message exchange is not available yet

EventDrivenPollingConsumer Options

The EventDrivePollingConsumer (the default implementation) supports the following options:

confluenceTableSmall

Option

Default

Description

pollingConsumerQueueSize

1000

Camel 2.14/2.13.1/2.12.4: The queue size for the internal hand-off queue between the polling consumer, and producers sending data into the queue.

pollingConsumerBlockWhenFull

true

Camel 2.14/2.13.1/2.12/4: Whether to block any producer if the internal queue is full.

pollingConsumerBlockTimeout0Camel 2.16: To use a timeout (in milliseconds) when the producer is blocked if the internal queue is full. If the value is 0 or negative then no timeout is in use. If a timeout is triggered then a ExchangeTimedOutException is thrown.

Notice that some Camel Components has their own implementation of PollingConsumer and therefore do not support the options above.

You can configure these options in endpoints URIs, such as shown below:

javaEndpoint endpoint = context.getEndpoint("file:inbox?pollingConsumerQueueSize=50"); PollingConsumer consumer = endpoint.createPollingConsumer(); Exchange exchange = consumer.receive(5000);

ConsumerTemplate

The ConsumerTemplate is a template much like Spring's JmsTemplate or JdbcTemplate supporting the Polling Consumer EIP. With the template you can consume Exchanges from an Endpoint. The template supports the three operations listed above. However, it also includes convenient methods for returning the body, etc consumeBody.

Example:

Exchange exchange = consumerTemplate.receive("activemq:my.queue");

Or to extract and get the body you can do:

Object body = consumerTemplate.receiveBody("activemq:my.queue");

And you can provide the body type as a parameter and have it returned as the type:

String body = consumerTemplate.receiveBody("activemq:my.queue", String.class);

You get hold of a ConsumerTemplate from the CamelContext with the createConsumerTemplate operation:

ConsumerTemplate consumer = context.createConsumerTemplate();

Using ConsumerTemplate with Spring DSL

With the Spring DSL we can declare the consumer in the CamelContext with the consumerTemplate tag, just like the ProducerTemplate. The example below illustrates this:{snippet:id=e1|lang=xml|url=camel/components/camel-spring/src/test/resources/org/apache/camel/spring/SpringConsumerTemplateTest-context.xml}Then we can get leverage Spring to inject the ConsumerTemplate in our java class. The code below is part of an unit test but it shows how the consumer and producer can work together.{snippet:id=e1|lang=java|url=camel/components/camel-spring/src/test/java/org/apache/camel/spring/SpringConsumerTemplateTest.java}

Timer Based Polling Consumer

In this sample we use a Timer to schedule a route to be started every 5th second and invoke our bean MyCoolBean where we implement the business logic for the Polling Consumer. Here we want to consume all messages from a JMS queue, process the message and send them to the next queue.

First we setup our route as:{snippet:id=e1|lang=java|url=camel/tags/camel-2.6.0/components/camel-jms/src/test/java/org/apache/camel/component/jms/JmsTimerBasedPollingConsumerTest.java}And then we have out logic in our bean:{snippet:id=e2|lang=java|url=camel/tags/camel-2.6.0/components/camel-jms/src/test/java/org/apache/camel/component/jms/JmsTimerBasedPollingConsumerTest.java}

Scheduled Poll Components

Quite a few inbound Camel endpoints use a scheduled poll pattern to receive messages and push them through the Camel processing routes. That is to say externally from the client the endpoint appears to use an Event Driven Consumer but internally a scheduled poll is used to monitor some kind of state or resource and then fire message exchanges.

Since this a such a common pattern, polling components can extend the ScheduledPollConsumer base class which makes it simpler to implement this pattern. There is also the Quartz Component which provides scheduled delivery of messages using the Quartz enterprise scheduler.

For more details see:

ScheduledPollConsumer Options

The ScheduledPollConsumer supports the following options:

confluenceTableSmall

Option

Default

Description

backoffErrorThreshold

0

Camel 2.12: The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

backoffIdleThreshold

0

Camel 2.12: The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

backoffMultiplier

0

Camel 2.12: To let the scheduled polling consumer back-off if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

delay

500

Milliseconds before the next poll.

greedy

false

Camel 2.10.6/2.11.1: If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.

initialDelay

1000

Milliseconds before the first poll starts.

pollStrategy

 

A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange has been created and routed in Camel. In other words the error occurred while the polling was gathering information, for instance access to a file network failed so Camel cannot access it to scan for files.

The default implementation will log the caused exception at WARN level and ignore it.

runLoggingLevel

TRACE

Camel 2.8: The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

scheduledExecutorService

null

Camel 2.10: Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. This option allows you to share a thread pool among multiple consumers.

scheduler

null

Camel 2.12: Allow to plugin a custom org.apache.camel.spi.ScheduledPollConsumerScheduler to use as the scheduler for firing when the polling consumer runs. The default implementation uses the ScheduledExecutorService and there is a Quartz2, and Spring based which supports CRON expressions. Notice: If using a custom scheduler then the options for initialDelay, useFixedDelay, timeUnit and scheduledExecutorService may not be in use. Use the text quartz2 to refer to use the Quartz2 scheduler; and use the text spring to use the Spring based; and use the text #myScheduler to refer to a custom scheduler by its id in the Registry.

See Quartz2 page for an example.

scheduler.xxx

null

Camel 2.12: To configure additional properties when using a custom scheduler or any of the Quartz2, Spring based scheduler.

sendEmptyMessageWhenIdle

false

Camel 2.9: If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

startScheduler

true

Whether the scheduler should be auto started.

timeUnit

TimeUnit.MILLISECONDS

Time unit for initialDelay and delay options.

useFixedDelay

 

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. In Camel 2.7.x or older the default value is false.

From Camel 2.8: the default value is true.

Using backoff to Let the Scheduler be Less Aggressive

Available as of Camel 2.12

The scheduled Polling Consumer is by default static by using the same poll frequency whether or not there is messages to pickup or not.

From Camel 2.12: you can configure the scheduled Polling Consumer to be more dynamic by using backoff. This allows the scheduler to skip N number of polls when it becomes idle, or there has been X number of errors in a row. See more details in the table above for the backoffXXX options.

For example to let a FTP consumer back-off if its becoming idle for a while you can do:

javafrom("ftp://myserver?username=foo&passowrd=secret?delete=true&delay=5s&backoffMultiplier=6&backoffIdleThreshold=5") .to("bean:processFile");

In this example, the FTP consumer will poll for new FTP files every 5th second. But if it has been idle for 5 attempts in a row, then it will back-off using a multiplier of 6, which means it will now poll every 5 x 6 = 30th second instead. When the consumer eventually pickup a file, then the back-off will reset, and the consumer will go back and poll every 5th second again.

Camel will log at DEBUG level using org.apache.camel.impl.ScheduledPollConsumer when back-off is kicking-in.

About Error Handling and Scheduled Polling Consumers

ScheduledPollConsumer is scheduled based and its run method is invoked periodically based on schedule settings. But errors can also occur when a poll is being executed. For instance if Camel should poll a file network, and this network resource is not available then a java.io.IOException could occur. As this error happens before any Exchange has been created and prepared for routing, then the regular Error handling in Camel does not apply. So what does the consumer do then? Well the exception is propagated back to the run method where its handled. Camel will by default log the exception at WARN level and then ignore it. At next schedule the error could have been resolved and thus being able to poll the endpoint successfully.

Using a Custom Scheduler

Available as of Camel 2.12:

The SPI interface org.apache.camel.spi.ScheduledPollConsumerScheduler allows to implement a custom scheduler to control when the Polling Consumer runs. The default implementation is based on the JDKs ScheduledExecutorService with a single thread in the thread pool. There is a CRON based implementation in the Quartz2, and Spring components.

For an example of developing and using a custom scheduler, see the unit test org.apache.camel.component.file.FileConsumerCustomSchedulerTest from the source code in camel-core.

Error Handling When Using PollingConsumerPollStrategy

org.apache.camel.PollingConsumerPollStrategy is a pluggable strategy that you can configure on the ScheduledPollConsumer. The default implementation org.apache.camel.impl.DefaultPollingConsumerPollStrategy will log the caused exception at WARN level and then ignore this issue.

The strategy interface provides the following three methods:

  • begin
    • void begin(Consumer consumer, Endpoint endpoint)
  • begin (Camel 2.3)
    • boolean begin(Consumer consumer, Endpoint endpoint)
  • commit
    • void commit(Consumer consumer, Endpoint endpoint)
  • commit (Camel 2.6)
    • void commit(Consumer consumer, Endpoint endpoint, int polledMessages)
  • rollback
    • boolean rollback(Consumer consumer, Endpoint endpoint, int retryCounter, Exception e) throws Exception

In Camel 2.3: the begin method returns a boolean which indicates whether or not to skipping polling. So you can implement your custom logic and return false if you do not want to poll this time.

In Camel 2.6: the commit method has an additional parameter containing the number of message that was actually polled. For example if there was no messages polled, the value would be zero, and you can react accordingly.

The most interesting is the rollback as it allows you do handle the caused exception and decide what to do.

For instance if we want to provide a retry feature to a scheduled consumer we can implement the PollingConsumerPollStrategy method and put the retry logic in the rollback method. Lets just retry up till three times:

javapublic boolean rollback(Consumer consumer, Endpoint endpoint, int retryCounter, Exception e) throws Exception { if (retryCounter < 3) { // return true to tell Camel that it should retry the poll immediately return true; } // okay we give up do not retry anymore return false; }

Notice that we are given the Consumer as a parameter. We could use this to restart the consumer as we can invoke stop and start:

java// error occurred lets restart the consumer, that could maybe resolve the issue consumer.stop(); consumer.start();

Note: if you implement the begin operation make sure to avoid throwing exceptions as in such a case the poll operation is not invoked and Camel will invoke the rollback directly.

Configuring an Endpoint to Use PollingConsumerPollStrategy

To configure an Endpoint to use a custom PollingConsumerPollStrategy you use the option pollStrategy. For example in the file consumer below we want to use our custom strategy defined in the Registry with the bean id myPoll:

from("file://inbox/?pollStrategy=#myPoll") .to("activemq:queue:inbox")

Using This Pattern

See Also

Competing Consumers

Camel supports the Competing Consumers from the EIP patterns using a few different components.

You can use the following components to implement competing consumers:-

  • Seda for SEDA based concurrent processing using a thread pool
  • JMS for distributed SEDA based concurrent processing with queues which support reliable load balancing, failover and clustering.

Enabling Competing Consumers with JMS

To enable Competing Consumers you just need to set the concurrentConsumers property on the JMS endpoint.

For example

from("jms:MyQueue?concurrentConsumers=5").bean(SomeBean.class);

or in Spring DSL

<route>
  <from uri="jms:MyQueue?concurrentConsumers=5"/>
  <to uri="bean:someBean"/>
</route>

Or just run multiple JVMs of any ActiveMQ or JMS route (smile)

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Message Dispatcher

Camel supports the Message Dispatcher from the EIP patterns using various approaches.

You can use a component like JMS with selectors to implement a Selective Consumer as the Message Dispatcher implementation. Or you can use an Endpoint as the Message Dispatcher itself and then use a Content Based Router as the Message Dispatcher.

 

Example

The following example demonstrates Message Dispatcher pattern using the Competing Consumers functionality of the JMS component to offload messages to a Content Based Router and custom Processors registered in the Camel Registry running in separate threads from originating consumer.

 

Using the Fluent Builders

from("jms:queue:foo?concurrentConsumers=5")
	.threads(5)
	.choice()
		.when(header("type").isEqualTo("A")) 
			.processRef("messageDispatchProcessorA")
		.when(header("type").isEqualTo("B"))
			.processRef("messageDispatchProcessorB")
		.when(header("type").isEqualTo("C"))
			.processRef("messageDispatchProcessorC")		
		.otherwise()
			.to("jms:queue:invalidMessageType");

 

Using the Spring XML Extensions

<route>
	<from uri="jms:queue:foo?concurrentConsumers=5"/>
	<threads poolSize="5">
		<choice>
			<when>
				<simple>${in.header.type} == 'A'</simple>
				<to ref="messageDispatchProcessorA"/>
			</when>
			<when>
				<simple>${in.header.type} == 'B'</simple>
				<to ref="messageDispatchProcessorB"/>
			</when>
			<when>
				<simple>${in.header.type} == 'C'</simple>
				<to ref="messageDispatchProcessorC"/>
			</when>
			<otherwise>
				<to uri="jms:queue:invalidMessageType"/>
		</choice>
	</threads>
</route>

See Also

 

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Selective Consumer

The Selective Consumer from the EIP patterns can be implemented in two ways

The first solution is to provide a Message Selector to the underlying URIs when creating your consumer. For example when using JMS you can specify a selector parameter so that the message broker will only deliver messages matching your criteria.

The other approach is to use a Message Filter which is applied; then if the filter matches the message your consumer is invoked as shown in the following example

Using the Fluent Builders

Error formatting macro: snippet: java.lang.NullPointerException

Using the Spring XML Extensions

Error formatting macro: snippet: java.lang.NullPointerException

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Durable Subscriber

Camel supports the Durable Subscriber from the EIP patterns using the JMS component which supports publish & subscribe using Topics with support for non-durable and durable subscribers.

Another alternative is to combine the Message Dispatcher or Content Based Router with File or JPA components for durable subscribers then something like Seda for non-durable.

Here is a simple example of creating durable subscribers to a JMS topic

Using the Fluent Builders

from("direct:start").to("activemq:topic:foo");

from("activemq:topic:foo?clientId=1&durableSubscriptionName=bar1").to("mock:result1");

from("activemq:topic:foo?clientId=2&durableSubscriptionName=bar2").to("mock:result2");

Using the Spring XML Extensions

<route>
    <from uri="direct:start"/>
    <to uri="activemq:topic:foo"/>
</route>

<route>
    <from uri="activemq:topic:foo?clientId=1&durableSubscriptionName=bar1"/>
    <to uri="mock:result1"/>
</route>

<route>
    <from uri="activemq:topic:foo?clientId=2&durableSubscriptionName=bar2"/>
    <to uri="mock:result2"/>
</route>

Here is another example of JMS durable subscribers, but this time using virtual topics (recommended by AMQ over durable subscriptions)

Using the Fluent Builders

from("direct:start").to("activemq:topic:VirtualTopic.foo");

from("activemq:queue:Consumer.1.VirtualTopic.foo").to("mock:result1");

from("activemq:queue:Consumer.2.VirtualTopic.foo").to("mock:result2");

Using the Spring XML Extensions

<route>
    <from uri="direct:start"/>
    <to uri="activemq:topic:VirtualTopic.foo"/>
</route>

<route>
    <from uri="activemq:queue:Consumer.1.VirtualTopic.foo"/>
    <to uri="mock:result1"/>
</route>

<route>
    <from uri="activemq:queue:Consumer.2.VirtualTopic.foo"/>
    <to uri="mock:result2"/>
</route>

See Also

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Idempotent Consumer

The Idempotent Consumer from the EIP patterns is used to filter out duplicate messages.

This pattern is implemented using the IdempotentConsumer class. This uses an Expression to calculate a unique message ID string for a given message exchange; this ID can then be looked up in the IdempotentRepository to see if it has been seen before; if it has the message is consumed; if its not then the message is processed and the ID is added to the repository.

The Idempotent Consumer essentially acts like a Message Filter to filter out duplicates.

Camel will add the message id eagerly to the repository to detect duplication also for Exchanges currently in progress.
On completion Camel will remove the message id from the repository if the Exchange failed, otherwise it stays there.

Camel provides the following Idempotent Consumer implementations:

Options

The Idempotent Consumer has the following options:

Option

Default

Description

eager

true

Eager controls whether Camel adds the message to the repository before or after the exchange has been processed. If enabled before then Camel will be able to detect duplicate messages even when messages are currently in progress. By disabling Camel will only detect duplicates when a message has successfully been processed.

messageIdRepositoryRef

null

A reference to a IdempotentRepository to lookup in the registry. This option is mandatory when using XML DSL.

skipDuplicate

true

Camel 2.8: Sets whether to skip duplicate messages. If set to false then the message will be continued. However the Exchange has been marked as a duplicate by having the Exchange.DUPLICATE_MESSAG exchange property set to a Boolean.TRUE value.

removeOnFailure

true

Camel 2.9: Sets whether to remove the id of an Exchange that failed.

completionEagerfalse

Camel 2.16: Sets whether to complete the idempotent consumer eager or when the exchange is done.

If this option is true to complete eager, then the idempotent consumer will trigger its completion when the exchange reached the end of the block of the idempotent consumer pattern. So if the exchange is continued routed after the block ends, then whatever happens there does not affect the state.

If this option is false (default) to not complete eager, then the idempotent consumer will complete when the exchange is done being routed. So if the exchange is continued routed after the block ends, then whatever happens there also affect the state. For example if the exchange failed due to an exception, then the state of the idempotent consumer will be a rollback.

Using the Fluent Builders

The following example will use the header myMessageId to filter out duplicates{snippet:id=idempotent|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/RouteBuilderTest.java}The above example will use an in-memory based MessageIdRepository which can easily run out of memory and doesn't work in a clustered environment. So you might prefer to use the JPA based implementation which uses a database to store the message IDs which have been processed{snippet:id=idempotent|lang=java|url=camel/trunk/components/camel-jpa/src/test/java/org/apache/camel/processor/jpa/JpaIdempotentConsumerTest.java}In the above example we are using the header messageId to filter out duplicates and using the collection myProcessorName to indicate the Message ID Repository to use. This name is important as you could process the same message by many different processors; so each may require its own logical Message ID Repository.

For further examples of this pattern in use you could look at the junit test case

Spring XML example

The following example will use the header myMessageId to filter out duplicates{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringIdempotentConsumerTest.xml}

How to handle duplicate messages in the route

Available as of Camel 2.8

You can now set the skipDuplicate option to false which instructs the idempotent consumer to route duplicate messages as well. However the duplicate message has been marked as duplicate by having a property on the Exchange set to true. We can leverage this fact by using a Content Based Router or Message Filter to detect this and handle duplicate messages.

For example in the following example we use the Message Filter to send the message to a duplicate endpoint, and then stop continue routing that message.{snippet:id=e1|lang=java|title=Filter duplicate messages|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/IdempotentConsumerTest.java}The sample example in XML DSL would be:{snippet:id=e1|lang=xml|title=Filter duplicate messages|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringIdempotentConsumerNoSkipDuplicateFilterTest.xml}

How to handle duplicate message in a clustered environment with a data grid

Available as of Camel 2.8

If you have running Camel in a clustered environment, a in memory idempotent repository doesn't work (see above). You can setup either a central database or use the idempotent consumer implementation based on the Hazelcast data grid. Hazelcast finds the nodes over multicast (which is default - configure Hazelcast for tcp-ip) and creates automatically a map based repository:

java HazelcastIdempotentRepository idempotentRepo = new HazelcastIdempotentRepository("myrepo"); from("direct:in").idempotentConsumer(header("messageId"), idempotentRepo).to("mock:out");

You have to define how long the repository should hold each message id (default is to delete it never). To avoid that you run out of memory you should create an eviction strategy based on the Hazelcast configuration. For additional information see camel-hazelcast.

See this little tutorial, how setup such an idempotent repository on two cluster nodes using Apache Karaf.

Available as of Camel 2.13.0

Another option for using Idempotent Consumer in a clustered environment is Infinispan. Infinispan is a data grid with replication and distribution clustering support. For additional information see camel-infinispan.

Using This Pattern

Transactional Client

Camel recommends supporting the Transactional Client from the EIP patterns using spring transactions.

Transaction Oriented Endpoints like JMS support using a transaction for both inbound and outbound message exchanges. Endpoints that support transactions will participate in the current transaction context that they are called from.

Configuration of Redelivery

The redelivery in transacted mode is not handled by Camel but by the backing system (the transaction manager). In such cases you should resort to the backing system how to configure the redelivery.

You should use the SpringRouteBuilder to setup the routes since you will need to setup the spring context with the TransactionTemplates that will define the transaction manager configuration and policies.

For inbound endpoint to be transacted, they normally need to be configured to use a Spring PlatformTransactionManager. In the case of the JMS component, this can be done by looking it up in the spring context.

You first define needed object in the spring configuration.

xml <bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager"> <property name="connectionFactory" ref="jmsConnectionFactory" /> </bean> <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory"> <property name="brokerURL" value="tcp://localhost:61616"/> </bean>

Then you look them up and use them to create the JmsComponent.

java PlatformTransactionManager transactionManager = (PlatformTransactionManager) spring.getBean("jmsTransactionManager"); ConnectionFactory connectionFactory = (ConnectionFactory) spring.getBean("jmsConnectionFactory"); JmsComponent component = JmsComponent.jmsComponentTransacted(connectionFactory, transactionManager); component.getConfiguration().setConcurrentConsumers(1); ctx.addComponent("activemq", component);

Transaction Policies

Outbound endpoints will automatically enlist in the current transaction context. But what if you do not want your outbound endpoint to enlist in the same transaction as your inbound endpoint? The solution is to add a Transaction Policy to the processing route. You first have to define transaction policies that you will be using. The policies use a spring TransactionTemplate under the covers for declaring the transaction demarcation to use. So you will need to add something like the following to your spring xml:

xml <bean id="PROPAGATION_REQUIRED" class="org.apache.camel.spring.spi.SpringTransactionPolicy"> <property name="transactionManager" ref="jmsTransactionManager"/> </bean> <bean id="PROPAGATION_REQUIRES_NEW" class="org.apache.camel.spring.spi.SpringTransactionPolicy"> <property name="transactionManager" ref="jmsTransactionManager"/> <property name="propagationBehaviorName" value="PROPAGATION_REQUIRES_NEW"/> </bean>

Then in your SpringRouteBuilder, you just need to create new SpringTransactionPolicy objects for each of the templates.

javapublic void configure() { ... Policy requried = bean(SpringTransactionPolicy.class, "PROPAGATION_REQUIRED")); Policy requirenew = bean(SpringTransactionPolicy.class, "PROPAGATION_REQUIRES_NEW")); ... }

Once created, you can use the Policy objects in your processing routes:

java // Send to bar in a new transaction from("activemq:queue:foo").policy(requirenew).to("activemq:queue:bar"); // Send to bar without a transaction. from("activemq:queue:foo").policy(notsupported ).to("activemq:queue:bar");

OSGi Blueprint

If you are using OSGi Blueprint then you most likely have to explicit declare a policy and refer to the policy from the transacted in the route.

xml <bean id="required" class="org.apache.camel.spring.spi.SpringTransactionPolicy"> <property name="transactionManager" ref="jmsTransactionManager"/> <property name="propagationBehaviorName" value="PROPAGATION_REQUIRED"/> </bean>

And then refer to "required" from the route:

xml<route> <from uri="activemq:queue:foo"/> <transacted ref="required"/> <to uri="activemq:queue:bar"/> </route>

Database Sample

In this sample we want to ensure that two endpoints is under transaction control. These two endpoints inserts data into a database.
The sample is in its full as a unit test.

First of all we setup the usual spring stuff in its configuration file. Here we have defined a DataSource to the HSQLDB and a most importantly the Spring DataSource TransactionManager that is doing the heavy lifting of ensuring our transactional policies. You are of course free to use any of the Spring based TransactionManager, eg. if you are in a full blown J2EE container you could use JTA or the WebLogic or WebSphere specific managers.

As we use the new convention over configuration we do not need to configure a transaction policy bean, so we do not have any PROPAGATION_REQUIRED beans. All the beans needed to be configured is standard Spring beans only, eg. there are no Camel specific configuration at all.{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/interceptor/springTransactionalClientDataSourceMinimalConfiguration.xml}Then we are ready to define our Camel routes. We have two routes: 1 for success conditions, and 1 for a forced rollback condition.
This is after all based on a unit test. Notice that we mark each route as transacted using the transacted tag.{snippet:id=e2|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/interceptor/springTransactionalClientDataSourceMinimalConfiguration.xml}That is all that is needed to configure a Camel route as being transacted. Just remember to use the transacted DSL. The rest is standard Spring XML to setup the transaction manager.

JMS Sample

In this sample we want to listen for messages on a queue and process the messages with our business logic java code and send them along. Since its based on a unit test the destination is a mock endpoint.

First we configure the standard Spring XML to declare a JMS connection factory, a JMS transaction manager and our ActiveMQ component that we use in our routing.{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-jms/src/test/resources/org/apache/camel/component/jms/tx/TransactionMinimalConfigurationTest.xml}And then we configure our routes. Notice that all we have to do is mark the route as transacted using the transacted tag.{snippet:id=e2|lang=xml|url=camel/trunk/components/camel-jms/src/test/resources/org/apache/camel/component/jms/tx/TransactionMinimalConfigurationTest.xml}

Transaction error handler

When a route is marked as transacted using transacted Camel will automatic use the TransactionErrorHandler as Error Handler. It supports basically the same feature set as the DefaultErrorHandler, so you can for instance use Exception Clause as well.

Integration Testing with Spring

An Integration Test here means a test runner class annotated @RunWith(SpringJUnit4ClassRunner.class).

When following the Spring Transactions documentation it is tempting to annotate your integration test with @Transactional then seed your database before firing up the route to be tested and sending a message in. This is incorrect as Spring will have an in-progress transaction, and Camel will wait on this before proceeding, leading to the route timing out.

Instead, remove the @Transactional annotation from the test method and seed the test data within a TransactionTemplate execution which will ensure the data is committed to the database before Camel attempts to pick up and use the transaction manager. A simple example can be found on GitHub.

Spring's transactional model ensures each transaction is bound to one thread. A Camel route may invoke additional threads which is where the blockage may occur. This is not a fault of Camel but as the programmer you must be aware of the consequences of beginning a transaction in a test thread and expecting a separate thread created by your Camel route to be participate, which it cannot. You can, in your test, mock the parts that cause separate threads to avoid this issue.

Using multiple routes with different propagation behaviors

Available as of Camel 2.2
Suppose you want to route a message through two routes and by which the 2nd route should run in its own transaction. How do you do that? You use propagation behaviors for that where you configure it as follows:

  • The first route use PROPAGATION_REQUIRED
  • The second route use PROPAGATION_REQUIRES_NEW

This is configured in the Spring XML file:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/interceptor/MixedTransactionPropagationTest.xml}Then in the routes you use transacted DSL to indicate which of these two propagations it uses.{snippet:id=e1|lang=java|url=camel/trunk/components/camel-spring/src/test/java/org/apache/camel/spring/interceptor/MixedTransactionPropagationTest.java}Notice how we have configured the onException in the 2nd route to indicate in case of any exceptions we should handle it and just rollback this transaction. This is done using the markRollbackOnlyLast which tells Camel to only do it for the current transaction and not globally.

See Also

Using This Pattern

Messaging Gateway

Camel has several endpoint components that support the Messaging Gateway from the EIP patterns.

Components like Bean and CXF provide a a way to bind a Java interface to the message exchange.

However you may want to read the Using CamelProxy documentation as a true Messaging Gateway EIP solution.
Another approach is to use @Produce which you can read about in POJO Producing which also can be used as a Messaging Gateway EIP solution.

 

Example

The following example how the CXF and Bean components can be used to abstract the developer from the underlying messaging system API


Using the Fluent Builders

from("cxf:bean:soapMessageEndpoint")
	.to("bean:testBean?method=processSOAP");

 

Using the Spring XML Extensions

<route>
	<from uri="cxf:bean:soapMessageEndpoint"/>
	<to uri="bean:testBean?method=processSOAP"/>
</route>

See Also

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Service Activator

Camel has several endpoint components that support the Service Activator from the EIP patterns.

Components like Bean, CXF and Pojo provide a a way to bind the message exchange to a Java interface/service where the route defines the endpoints and wires it up to the bean.

In addition you can use the Bean Integration to wire messages to a bean using annotation.

Here is a simple example of using a Direct endpoint to create a messaging interface to a Pojo Bean service.

Using the Fluent Builders

from("direct:invokeMyService").to("bean:myService");

Using the Spring XML Extensions

<route>
    <from uri="direct:invokeMyService"/>
    <to uri="bean:myService"/>
</route>

See Also

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

System Management

Detour

The Detour from the EIP patterns allows you to send messages through additional steps if a control condition is met. It can be useful for turning on extra validation, testing, debugging code when needed.

Example

In this example we essentially have a route like from("direct:start").to("mock:result") with a conditional detour to the mock:detour endpoint in the middle of the route..

Error formatting macro: snippet: java.lang.NullPointerException

Using the Spring XML Extensions

<route>
  <from uri="direct:start"/>
    <choice>
      <when>
        <method bean="controlBean" method="isDetour"/>
	<to uri="mock:detour"/>
      </when>
    </choice>
    <to uri="mock:result"/>
</route>

whether the detour is turned on or off is decided by the ControlBean. So, when the detour is on the message is routed to mock:detour and then mock:result. When the detour is off, the message is routed to mock:result.

For full details, check the example source here:

camel-core/src/test/java/org/apache/camel/processor/DetourTest.java

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Wire Tap

Wire Tap (from the EIP patterns) allows you to route messages to a separate location while they are being forwarded to the ultimate destination.

Streams

If you Wire Tap a stream message body then you should consider enabling Stream caching to ensure the message body can be read at each endpoint. See more details at Stream caching.

Options

Name

Default

Description

uri

 

Mandatory: The URI of the endpoint to which the wire-tapped message should be sent.

From Camel 2.16: support for dynamic to URIs is as documented in Message Endpoint.

executorServiceRef

 

Reference ID of a custom Thread Pool to use when processing the wire-tapped messages.

When not set, Camel will use an instance of the default thread pool.

processorRef

 

Reference ID of a custom Processor to use for creating a new message.

See "Sending a New Exchange" below.

copy

true

Camel 2.3: Whether to copy the Exchange before wire-tapping the message.

onPrepareRef

 

Camel 2.8: Reference identifier of a custom Processor to prepare the copy of the Exchange to be wire-tapped. This allows you to do any custom logic, such as deep-cloning the message payload.

cacheSize

 

Camel 2.16: Allows to configure the cache size for the ProducerCache which caches producers for reuse. Will by default use the default cache size which is 1000.

Setting the value to -1 allows to turn off the cache all together.

ignoreInvalidEndpoint

false

Camel 2.16: Whether to ignore an endpoint URI that could not be resolved.

When false, Camel will throw an exception when it identifies an invalid endpoint URI.

WireTap Threadpool

The Wire Tap uses a thread pool to process the tapped messages. This thread pool will by default use the settings detailed at Threading Model. In particular, when the pool is exhausted (with all threads utilized), further wiretaps will be executed synchronously by the calling thread. To remedy this, you can configure an explicit thread pool on the Wire Tap having either a different rejection policy, a larger worker queue, or more worker threads.

WireTap Node

Camel's Wire Tap node supports two flavors when tapping an Exchange:

  • With the traditional Wire Tap, Camel will copy the original Exchange and set its Exchange Pattern to InOnly, as we want the tapped Exchange to be sent in a fire and forget style. The tapped Exchange is then sent in a separate thread so it can run in parallel with the original. Beware that only the Exchange is copied - Wire Tap won't do a deep clone (unless you specify a custom processor via onPrepareRef which does that). So all copies could share objects from the original Exchange.
  • Camel also provides an option of sending a new Exchange allowing you to populate it with new values.

Sending a Copy (traditional wiretap)

Using the Fluent Builders{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/WireTapTest.java}Using the Spring XML Extensions{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringWireTapTest.xml}

Sending a New Exchange

Using the Fluent Builders
Camel supports either a processor or an Expression to populate the new Exchange. Using a processor gives you full power over how the Exchange is populated as you can set properties, headers, etc. An Expression can only be used to set the IN body.

From Camel 2.3: the Expression or Processor is pre-populated with a copy of the original Exchange, which allows you to access the original message when you prepare a new Exchange to be sent. You can use the copy option (enabled by default) to indicate whether you want this. If you set copy=false, then it works as in Camel 2.2 or older where the Exchange will be empty.

Below is the processor variation. This example is from Camel 2.3, where we disable copy by passing in false to create a new, empty Exchange.{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/WireTapUsingFireAndForgetTest.java}Here is the Expression variation. In the following example we disable copying by setting copy=false which results in the creation of a new, empty Exchange.{snippet:id=e2|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/WireTapUsingFireAndForgetTest.java}Using the Spring XML Extensions
The processor variation, which uses a processorRef attribute to refer to a Spring bean by ID:{snippet:id=e2|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringWireTapUsingFireAndForgetTest.xml}Here is the Expression variation, where the expression is defined in the body tag:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringWireTapUsingFireAndForgetTest.xml}This variation accesses the body of the original message and creates a new Exchange based on the Expression. It will create a new Exchange and have the body contain "Bye ORIGINAL BODY MESSAGE HERE"{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringWireTapUsingFireAndForgetCopyTest.xml}

Further Example

For another example of this pattern, refer to the wire tap test case.

Using Dynamic URIs

Available as of Camel 2.16:

For example to wire tap to a dynamic URI, then it supports the same dynamic URIs as documented in Message Endpoint. For example to wire tap to a JMS queue where the header ID is part of the queue name:

from("direct:start") .wireTap("jms:queue:backup-${header.id}") .to("bean:doSomething");

 

Sending a New Exchange and Set Headers in DSL

Available as of Camel 2.8

If you send a new message using Wire Tap, then you could only set the message body using an Expression from the DSL. If you also need to set headers, you would have to use a Processor. From Camel 2.8: it's possible to set headers as well using the DSL.

The following example sends a new message which has

  • Bye World as message body.
  • A header with key id with the value 123.
  • A header with key date which has current date as value.

Java DSL

{snippet:id=e1|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/WireTapNewExchangeTest.java}

XML DSL

The XML DSL is slightly different than Java DSL in how you configure the message body and headers using <body> and <setHeader>:{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringWireTapNewExchangeTest.xml}

Using onPrepare to Execute Custom Logic when Preparing Messages

Available as of Camel 2.8

See details at Multicast

Using This Pattern

Log

How can I log the processing of a Message?

Camel provides many ways to log the fact that you are processing a message. Here are just a few examples:

  • You can use the Log component which logs the Message content.
  • You can use the Tracer which trace logs message flow.
  • You can also use a Processor or Bean and log from Java code.
  • You can use the log DSL.

Using log DSL

In Camel 2.2 you can use the log DSL which allows you to use Simple language to construct a dynamic message which gets logged.
For example you can do

from("direct:start").log("Processing ${id}").to("bean:foo");

Which will construct a String message at runtime using the Simple language. The log message will by logged at INFO level using the route id as the log name. By default a route is named route-1, route-2 etc. But you can use the routeId("myCoolRoute") to set a route name of choice.

Difference between log in the DSL and [Log] component

The log DSL is much lighter and meant for logging human logs such as Starting to do ... etc. It can only log a message based on the Simple language. On the other hand Log component is a full fledged component which involves using endpoints and etc. The Log component is meant for logging the Message itself and you have many URI options to control what you would like to be logged.

Using Logger instance from the the Registry

As of Camel 2.12.4/2.13.1, if no logger name or logger instance is passed to log DSL, there is a Registry lookup performed to find single instance of org.slf4j.Logger. If such an instance is found, it is used instead of creating a new logger instance. If more instances are found, the behavior defaults to creating a new instance of logger.

Logging message body with streamed messages

If the message body is stream based, then logging the message body, may cause the message body to be empty afterwards. See this FAQ. For streamed messages you can use Stream caching to allow logging the message body and be able to read the message body afterwards again.

The log DSL have overloaded methods to set the logging level and/or name as well.

from("direct:start").log(LoggingLevel.DEBUG, "Processing ${id}").to("bean:foo");

and to set a logger name

from("direct:start").log(LoggingLevel.DEBUG, "com.mycompany.MyCoolRoute", "Processing ${id}").to("bean:foo");

Since Camel 2.12.4/2.13.1 the logger instance may be used as well:

from("direct:start").log(LoggingLeven.DEBUG, org.slf4j.LoggerFactory.getLogger("com.mycompany.mylogger"), "Processing ${id}").to("bean:foo");

For example you can use this to log the file name being processed if you consume files.

from("file://target/files").log(LoggingLevel.DEBUG, "Processing file ${file:name}").to("bean:foo");

Using log DSL from Spring

In Spring DSL it is also easy to use log DSL as shown below:

        <route id="foo">
            <from uri="direct:foo"/>
            <log message="Got ${body}"/>
            <to uri="mock:foo"/>
        </route>

The log tag has attributes to set the message, loggingLevel and logName. For example:

        <route id="baz">
            <from uri="direct:baz"/>
            <log message="Me Got ${body}" loggingLevel="FATAL" logName="com.mycompany.MyCoolRoute"/>
            <to uri="mock:baz"/>
        </route>

Since Camel 2.12.4/2.13.1 it is possible to reference logger instance. For example:

        <bean id="myLogger" class="org.slf4j.LoggerFactory" factory-method="getLogger" xmlns="http://www.springframework.org/schema/beans">
            <constructor-arg value="com.mycompany.mylogger" />
        </bean>

        <route id="moo" xmlns="http://camel.apache.org/schema/spring">
            <from uri="direct:moo"/>
            <log message="Me Got ${body}" loggingLevel="INFO" loggerRef="myLogger"/>
            <to uri="mock:baz"/>
        </route>

Configuring log name globally

Available as of Camel 2.17

By default the log name is the route id. If you want to use a different log name, you would need to configure the logName option. However if you have many logs and you want all of them to use the same log name, then you would need to set that logName option on all of them.

With Camel 2.17 onwards you can configure a global log name that is used instead of the route id, eg

CamelContext context = ...
context.getProperties().put(Exchange.LOG_EIP_NAME, "com.foo.myapp");

And in XML

<camelContext ...>
  <properties>
    <property key="CamelLogEipName" value="com.foo.myapp"/>
  </properties>

 

Using slf4j Marker

Available as of Camel 2.9

You can specify a marker name in the DSL

        <route id="baz">
            <from uri="direct:baz"/>
            <log message="Me Got ${body}" loggingLevel="FATAL" logName="com.mycompany.MyCoolRoute" marker="myMarker"/>
            <to uri="mock:baz"/>
        </route>

Using log DSL in OSGi

Improvement as of Camel 2.12.4/2.13.1

When using log DSL inside OSGi (e.g., in Karaf), the underlying logging mechanisms are provided by PAX logging. It searches for a bundle which invokes org.slf4j.LoggerFactory.getLogger() method and associates the bundle with the logger instance. Passing only logger name to log DSL results in associating camel-core bundle with the logger instance created.

In some scenarios it is required that the bundle associated with logger should be the bundle which contains route definition. This is possible using provided logger instance both for Java DSL and Spring DSL (see the examples above).

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.

Error rendering macro 'include'

null

  • No labels