Page History

...

• MemoryIdempotentRepository
• FileIdempotentRepository
• HazelcastIdempotentRepository (Available as of Camel 2.8)
• JdbcMessageIdRepository (Available as of Camel 2.7)
• JpaMessageIdRepository

• InfinispanIdempotentRepository (Available as of Camel 2.13.0)

• JCacheIdempotentRepository (Available as of Camel 2.17.0)

• SpringCacheIdempotentRepository (Available as of Camel 2.17.1)
  

• EhcacheIdempotentRepository (Available as of Camel 2.18.0)

• KafkaIdempotentRepository (Available as of Camel 2.19.0)

Options

The Idempotent Consumer has the following options:

...

The following example will use the header myMessageId to filter out duplicates Wiki Markup{snippet:id=idempotent|lang=java|url=camel/trunk/camel-core/src/test/java/org/apache/camel/builder/RouteBuilderTest.java}The above example will use an in-memory based MessageIdRepository which can easily run out of memory and doesn't work in a clustered environment. So you might prefer to use the JPA based implementation which uses a database to store the message IDs which have been processed Wiki Markup{snippet:id=idempotent|lang=java|url=camel/trunk/components/camel-jpa/src/test/java/org/apache/camel/processor/jpa/JpaIdempotentConsumerTest.java}In the above example we are using the header messageId to filter out duplicates and using the collection myProcessorName to indicate the Message ID Repository to use. This name is important as you could process the same message by many different processors; so each may require its own logical Message ID Repository.

...

The following example will use the header myMessageId to filter out duplicates Wiki Markup{snippet:id=e1|lang=xml|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringIdempotentConsumerTest.xml}

How to handle duplicate messages in the route

...

For example in the following example we use the Message Filter to send the message to a duplicate endpoint, and then stop continue routing that message. Wiki Markup{snippet:id=e1|lang=java|title=Filter duplicate messages|url=camel/trunk/camel-core/src/test/java/org/apache/camel/processor/IdempotentConsumerTest.java}The sample example in XML DSL would be: Wiki Markup{snippet:id=e1|lang=xml|title=Filter duplicate messages|url=camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/spring/processor/SpringIdempotentConsumerNoSkipDuplicateFilterTest.xml}

How to handle duplicate message in a clustered environment with a data grid

...

If you have running Camel in a clustered environment, a in memory idempotent repository doesn't work (see above). You can setup either a central database or use the idempotent consumer implementation based on the Hazelcast data grid. Hazelcast finds the nodes over multicast (which is default - configure Hazelcast for tcp-ip) and creates automatically a map based repository:

...

...

You have to define how long the repository should hold each message id (default is to delete it never). To avoid that you run out of memory you should create an eviction strategy based on the Hazelcast configuration. For additional information see camel-hazelcast.

...

Another option for using Idempotent Consumer in a clustered environment is Infinispan. Infinispan is a data grid with replication and distribution clustering support. For additional information seeUsing This Pattern camel-infinispan.
Include Page

Using This Pattern