You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Under Construction

This tutorial is a work in progress.

Background and Introduction

So there's a company, which we'll call Acme. Acme sells widgets, in a fairly unusual way. Their customers have stock rooms with tons of Acme widgets, owned by Acme. When the customer requires a widget, they take it out of the stock room. Then they enter into their own systems (ERP or whatever) that they bought the widget. Then at some point, their systems emit a record of the sale which needs to go to Acme so Acme can bill them for it. Obviously, everyone wants this to be as automated as possible, so there needs to be integration between the customer's system and Acme.

Sadly, Acme's sales people are, technically speaking, doormats. They tell all their prospects, "you can send us the data in whatever format, using whatever protocols, whatever. You just can't change once it's up and running."

The result is pretty much what you'd expect. Taking a random sample of 3 customers:

  • Customer 1: XML over FTP
  • Customer 2: CSV over HTTP
  • Customer 3: Excel via e-mail

Now on the Acme side, all this has to be converted to a canonical XML format and submitted to the Acme accounting system via JMS. Then the Acme accounting system sends an XML reply via JMS, with a summary of what it processed (e.g. 3 line items accepted, line item #2 in error, total invoice $123.45). Finally, that data needs to be formatted into an e-mail, and sent to a contact at the customer in question ("Dear Joyce, we received an invoice on 1/2/08. We accepted 3 line items totaling $123.45, though there was an error with line items #2 [invalid quantity ordered]. Thank you for your business. Love, Acme.").

So it turns out Camel can handle all this:

  • Listen for HTTP, e-mail, and FTP files
  • Grab attachments from the e-mail messages
  • Convert XML, XLS, and CSV files to a canonical XML format
    • using XSLT, automatic data format conversion, and/or POJOs that write to JAXB objects
  • read and write JMS messages
  • route based on company ID
  • format e-mails using Velocity templates
  • send outgoing e-mail messages

This tutorial will cover all that, plus setting up tests along the way.

High-Level Diagram

Here's more or less what the integration process looks like.

First, the input from the customers to Acme:

And then, the output from Acme to the customers:

Tutorial Tasks

Here's what we'll try to accomplish in this tutorial:

  1. Get sample files for the customer Excel, CSV, and XML input
  2. Get a sample file for the canonical XML format that Acme's accounting system uses
  3. Create an XSD for the canonical XML format
  4. Create JAXB POJOs corresponding to the canonical XSD
  5. Create an XSLT stylesheet to convert the Customer 1 (XML over FTP) messages to the canonical format
  6. Create a unit test to ensure that a simple Camel route invoking the XSLT stylesheet works
  7. Create a POJO that converts a List<List<String>> to the above JAXB POJOs
    • Note that Camel can automatically convert CSV input to a List of Lists of Strings representing the rows and columns of the CSV, so we'll use this POJO to handle Customer 2 (CSV over HTTP)
  8. Create a unit test to ensure that a simple Camel route invoking the CSV processing works
  9. Create a POJO that converts a Customer 3 Excel file to the above JAXB POJOs (using POI to read Excel)
  10. Create a unit test to ensure that a simple Camel route invoking the Excel processing works
  11. Create a POJO that reads an input message, takes an attachment off the message, and replaces the body of the message with the attachment
    • This is assuming for Customer 3 (Excel over e-mail) that the e-mail contains a single Excel file as an attachment, and the actual e-mail body is throwaway
  12. Build a set of Camel routes to handle the entire input (Customer -> Acme) side of the scenario.
  13. Build unit tests for the Camel input.
  14. TODO: Tasks for the output (Acme -> Customer) side of the scenario

Let's Get Started!

Step 1: Get Sample Files

You can make up your own if you like, but here are the "off the shelf" ones:

If you look at these files, you'll see that the different input formats use different field names and/or ordering, because of course the sales guys were totally OK with that. Sigh.

  • No labels