Page History

Tutorial Business Partners

...

• Customer 1: XML over FTP
• Customer 2: CSV over HTTP
• Customer 3: Excel via e-mail

...

Now on the Acme side, all this has to be converted to a canonical XML format and submitted to the Acme accounting system via JMS. Then the Acme accounting system does its stuff and sends an XML reply via JMS, with a summary of what it processed (e.g. 3 line items accepted, line item #2 in error, total invoice $123.45). Finally, that data needs to be formatted into an e-mail, and sent to a contact at the customer in question ("Dear Joyce, we received an invoice on 1/2/08. We accepted 3 line items totaling $123.45, though there was an error with line items #2 \ [invalid quantity ordered\]. Thank you for your business. Love, Acme.").

So it turns out Camel can handle all this:

...

This tutorial will cover all that, plus setting up tests along the way. You'll learn:

Before starting, you should be familiar with:

• Camel concepts including the CamelContext, Routes, Components and Endpoints, and Enterprise Integration Patterns
• Configuring Camel with the XML or Java DSL

You'll learn:

• How to set up a Maven build for a Camel project
• How to set up a Maven build for a Camel project
• How to transform XML, CSV, and Excel data into a standard XML format with Camel
  • How to write POJOs (Plain Old Java Objects), Velocity templates, and XSLT stylesheets that are invoked by Camel routes for message transformation
• How to configure simple and complex Routes in Camel, using either the XML or the Java DSL format
• How to set up unit tests that load a Camel configuration and test Camel routes
• How to use Camel's Data Formats to automatically convert data between Java objects and XML, CSV files, etc.
• How to send and receive e-mail from Camel
• How to send and receive JMS messages from Camel
• How to use Enterprise Integration Patterns including Message Router and Pipes and Filters
  • How to use various languages to express content-based routing rules in Camel
• How to deal with Camel messages, headers, and attachments

...

1. Add dependencies on Camel-Spring, and the Spring test JAR (which will automatically bring in JUnit 3.8.x) to your POM:

   Code Block
   xml xml
   <dependency> <artifactId>camel-spring</artifactId> <groupId>org.apache.camel</groupId> <version>1.4.0</version> </dependency> <dependency> <artifactId>spring-test</artifactId> <groupId>org.springframework</groupId> <version>2.5.5</version> <scope>test</scope> </dependency>

2. Create a new unit test class in src/test/java/your-package-here, perhaps called XMLInputTest.java
3. Make the test extend Spring's AbstractJUnit38SpringContextTests class, so it can load a Spring context for the test
4. Create a Spring context configuration file in src/test/resources, perhaps called XMLInputTest-context.xml
5. In the unit test class, use the class-level @ContextConfiguration annotation to indicate that a Spring context should be loaded
   • By default, this looks for a Context configuration file called TestClassName-context.xml in a subdirectory corresponding to the package of the test class. For instance, if your test class was org.apache.camel.tutorial.XMLInputTest, it would look for org/apache/camel/tutorial/XMLInputTest-context.xml
   • To override this default, use the locations attribute on the @ContextConfiguration annotation to provide specific context file locations (starting each path with a / if you don't want it to be relative to the package directory). My solution does this so I can put the context file directly in src/test/resources instead of in a package directory under there.
6. Add a CamelContext instance variable to the test class, with the @Autowired annotation. That way Spring will automatically pull the CamelContext out of the Spring context and inject it into our test class.
7. Add a ProducerTemplate instance variable and a setUp method that instantiates it from the CamelContext. We'll use the ProducerTemplate later to send messages to the route.

   Code Block
   java java
   protected ProducerTemplate<Exchange> template; protected void setUp() throws Exception { super.setUp(); template = camelContext.createProducerTemplate(); }

8. Put in an empty test method just for the moment (so when we run this we can see that "1 test succeeded")
9. Add the Spring <beans> element (including the Camel Namespace) with an empty <camelContext> element to the Spring context, like this:

   Code Block
   xml xml
   <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring-1.4.0.xsd"> <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring"> </camelContext> </beans>

...

1. Save the input-customer1.xml file to src/test/resources
2. Save your XSLT file (created in the previous step) to src/main/resources
3. Write a Camel Route, either right in the Spring XML, or using the Java DSL (in another class under src/test/java somewhere). This route should use the Pipes and Filters integration pattern to:
   1. Start from the endpoint direct:start (which lets the test conveniently pass messages into the route)
   2. Call the endpoint xslt:YourXSLTFile.xsl (to transform the message with the specified XSLT template)
   3. Send the result to the endpoint mock:finish (which lets the test verify the route output)
4. Add a test method to the unit test class that:
   1. Get a reference to the Mock endpoint mock:finish using code like this:

      Code Block
      java java
      MockEndpoint finish = MockEndpoint.resolve(camelContext, "mock:finish");

   2. Set the expectedMessageCount on that endpoint to 1
   3. Get a reference to the Customer 1 input file, using code like this:

      Code Block
      java java
      InputStream in = XMLInputTest.class.getResourceAsStream("/input-partner1.xml"); assertNotNull(in);

   4. Send that InputStream as a message to the direct:start endpoint, using code like this:

      Code Block
      java java
      template.sendBody("direct:start", in);

      Note that we can send the sample file body in several formats (File, InputStream, String, etc.) but in this case an InputStream is pretty convenient.
   5. Ensure that the message made it through the route to the final endpoint, by testing all configured Mock endpoints like this:

      Code Block
      java java
      MockEndpoint.assertIsSatisfied(camelContext);

   6. If you like, inspect the final message body using some code like finish.getExchanges().get(0).getIn().getBody().
      • If you do this, you'll need to know what format that body is – String, byte array, InputStream, etc.
5. Run your test with mvn install and make sure the build completes successfully.

...

1. Create a new class under src/main/java, perhaps called CSVConverterBean.
2. Add a method, with one argument of type List<List<String>> and the return type Invoice
   • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
3. In the method, the logic should look roughly like this:
   1. Create a new Invoice, using the method on the generated ObjectFactory class
   2. Loop through all the rows in the incoming CSV (the outer List)
   3. Skip the first row, which contains headers (column names)
   4. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values (the Strings in the inner List) and put them into the correct fields of the LineItemType
         • Not all of the values will actually go into the line item in this example
         • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
         • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means using a SimpleDateFormat to parse the date and setting that date on a GregorianCalendar
      3. Add the line item to the invoice
   5. Populate the partner ID, date of receipt, and order total on the Invoice
   6. Throw any exceptions out of the method, so Camel knows something went wrong
   7. Return the finished Invoice

...

Step 6: Initial Work on Customer 3 Input (Excel over e-mail)

Create an Excel-handling POJO

Create a unit test

...

/**
 * A test class the ensure we can convert Partner 2 CSV input files to the
 * canonical XML output format, using JAXB POJOs.
 */
@ContextConfiguration(locations = "/CSVInputTest-context.xml")
public class CSVInputTest extends AbstractJUnit38SpringContextTests {
    @Autowired
    protected CamelContext camelContext;
    protected ProducerTemplate<Exchange> template;

    protected void setUp() throws Exception {
        super.setUp();
        template = camelContext.createProducerTemplate();
    }

    public void testCSVConversion() {
        // TODO
    }
}

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                               http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://activemq.apache.org/camel/schema/spring
                               http://activemq.apache.org/camel/schema/spring/camel-spring-1.4.0.xsd">

    <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">
        <!-- TODO -->
    </camelContext>
</beans>

Now the meaty part is to flesh out the test class and write the Camel routes.

1. Update the Maven POM to include CSV Data Format support:

   Code Block
   xml xml
   <dependency> <artifactId>camel-csv</artifactId> <groupId>org.apache.camel</groupId> <version>1.4.0</version> </dependency>

2. Write the routes (right in the Spring XML context, or using the Java DSL) for the CSV conversion process, again using the Pipes and Filters pattern:
   1. Start from the endpoint direct:CSVstart (which lets the test conveniently pass messages into the route). We'll name this differently than the starting point for the previous test, in case you use the Java DSL and put all your routes in the same package (which would mean that each test would load the DSL routes for several tests.)
   2. This time, there's a little preparation to be done. Camel doesn't know that the initial input is a CSV, so it won't be able to convert it to the expected List<List<String>> without a little hint. For that, we need an unmarshal transformation in the route. The unmarshal method (in the DSL) or element (in the XML) takes a child indicating the format to unmarshal; in this case that should be csv.
   3. Next invoke the POJO to transform the message with a bean:CSVConverter endpoint
   4. As before, send the result to the endpoint mock:finish (which lets the test verify the route output)
   5. Finally, we need a Spring <bean> element in the Spring context XML file (but outside the <camelContext> element) to define the Spring bean that our route invokes. This Spring bean should have a name attribute that matches the name used in the bean endpoint (CSVConverter in the example above), and a class attribute that points to the CSV-to-JAXB POJO class you wrote above (such as, org.apache.camel.tutorial.CSVConverterBean). When Spring is in the picture, any bean endpoints look up Spring beans with the specified name.
3. Write a test method in the test class, which should look very similar to the previous test class:
   1. Get the MockEndpoint for the final endpoint, and tell it to expect one message
   2. Load the Partner 2 sample CSV file from the ClassPath, and send it as the body of a message to the starting endpoint
   3. Verify that the final MockEndpoint is satisfied (that is, it received one message) and examine the message body if you like
      • Note that we didn't marshal the JAXB POJOs to XML in this test, so the final message should contain an Invoice as the body. You could write a simple line of code to get the Exchange (and Message) from the MockEndpoint to confirm that.
4. Run this new test with mvn install and make sure it passes and the build completes successfully.

Solution: Your finished test might look something like this:

• src/test/java/org/apache/camel/tutorial/CSVInputTest.java
• For XML Configuration:
  • src/test/resources/CSVInputTest-context.xml
• Or, for Java DSL Configuration:
  • src/test/resources/CSVInputTest-dsl-context.xml
  • src/test/java/org/apache/camel/tutorial/routes/CSVInputTestRoute.java

Step 6: Initial Work on Customer 3 Input (Excel over e-mail)

To get a start on Customer 3, we'll create a POJO to convert the Customer 3 sample Excel data into the JAXB POJOs representing the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the Excel conversion and JAXB handling is valid and can be run safely in Camel.

Create an Excel-handling POJO

Camel does not have a data format handler for Excel by default. We have two options – create an Excel DataFormat (so Camel can convert Excel spreadsheets to something like the CSV List<List<String>> automatically), or create a POJO that can translate Excel data manually. For now, the second approach is easier (if we go the DataFormat route, we need code to both read and write Excel files, whereas otherwise read-only will do).

So, we need a POJO with a method that takes something like an InputStream or byte[] as an argument, and returns in Invoice as before. The process should look something like this:

1. Update the Maven POM to include POI support:

   Code Block
   xml xml
   <dependency> <artifactId>poi</artifactId> <groupId>org.apache.poi</groupId> <version>3.1-FINAL</version> </dependency>

2. Create a new class under src/main/java, perhaps called ExcelConverterBean.
3. Add a method, with one argument of type InputStream and the return type Invoice
   • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
4. In the method, the logic should look roughly like this:
   1. Create a new Invoice, using the method on the generated ObjectFactory class
   2. Create a new HSSFWorkbook from the InputStream, and get the first sheet from it
   3. Loop through all the rows in the sheet
   4. Skip the first row, which contains headers (column names)
   5. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values and put them into the correct fields of the LineItemType (you'll need some data type conversion logic)
         • Not all of the values will actually go into the line item in this example
         • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
         • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means setting the date from a date cell on a GregorianCalendar
      3. Add the line item to the invoice
   6. Populate the partner ID, date of receipt, and order total on the Invoice
   7. Throw any exceptions out of the method, so Camel knows something went wrong
   8. Return the finished Invoice

Solution: Here's an example of what the ExcelConverterBean might look like.

Create a unit test

The unit tests should be pretty familiar now. The test class and context for the Excel bean should be quite similar to the CSV bean.

1. Create the basic test class and corresponding Spring Context XML configuration file
2. The XML config should look a lot like the CSV test, except:
   • Remember to use a different start endpoint name if you're using the Java DSL and not use separate packages per test
   • You don't need the unmarshal step since the Excel POJO takes the raw InputStream from the source endpoint
   • You'll declare a <bean> and endpoint for the Excel bean prepared above instead of the CSV bean
3. The test class should look a lot like the CSV test, except use the right input file name and start endpoint name.

Solution: Your finished test might look something like this:

• src/test/java/org/apache/camel/tutorial/ExcelInputTest.java
• For XML Configuration:
  • src/test/resources/ExcelInputTest-context.xml
• Or, for Java DSL Configuration:
  • src/test/resources/ExcelInputTest-dsl-context.xml
  • src/test/java/org/apache/camel/tutorial/routes/ExcelInputTestRoute.java

Step 7: Put this all together into Camel routes for the Customer Input

With all the data type conversions working, the next step is to write the real routes that listen for HTTP, FTP, or e-mail input, and write the final XML output to an ActiveMQ queue. Along the way these routes will use the data conversions we've developed above.

So we'll create 3 routes to start with, as shown in the diagram back at the beginning:

1. Accept XML orders over FTP from Customer 1 (we'll assume the FTP server dumps files in a local directory on the Camel machine)
2. Accept CSV orders over HTTP from Customer 2
3. Accept Excel orders via e-mail from Customer 3 (we'll assume the messages are sent to an account we can access via IMAP)

...

Step 8: Create a unit test for the Customer Input Routes