...
Camel supports the Dead Letter Channel from the EIP patterns using the DeadLetterChannel processor which is an Error Handler.
...
title | Difference between Dead Letter Channel and Default Error Handler |
---|
The major difference is that Dead Letter Channel has a dead letter queue that whenever an Exchange could not be processed is moved to. It will always moved failed exchanges to this queue.
Unlike the Default Error Handler that does not have a dead letter queue. So whenever an Exchange could not be processed the error is propagated back to the client.
...
The DefaultErrorHandler
does very little: it ends the Exchange immediately and propagates the thrown Exception back to the caller.
The DeadLetterChannel
lets you control behaviors including redelivery, whether to propagate the thrown Exception to the caller (the handled
option), and where the (failed) Exchange should now be routed to.
The DeadLetterChannel
is also by default configured to not be verbose in the logs, so when a message is handled and moved to the dead letter endpoint, then there is nothing logged. If you want some level of logging you can use the various options on the redelivery policy / dead letter channel to configure this. For example if you want the message history then set logExhaustedMessageHistory=true
(and logHandled=true
for Camel 2.15.x or older).
When the DeadLetterChannel
moves a message to the dead letter endpoint, any new Exception thrown is by default handled by the dead letter channel as well. This ensures that the DeadLetterChannel
will always succeed. From Camel 2.15: this behavior can be changed by setting the option deadLetterHandleNewException=false
. Then if a new Exception is thrown, then the dead letter channel will fail and propagate back that new Exception (which is the behavior of the default error handler). When a new Exception occurs then the dead letter channel logs this at WARN
level. This can be turned off by setting logNewException=false
...
.
Redelivery
It is common for a temporary outage or database deadlock to cause a message to fail to process; but the chances are if its tried a few more times with some time delay then it will complete fine. So we typically wish to use some kind of redelivery policy to decide how many times to try redeliver a message and how long to wait before redelivery attempts.
The RedeliveryPolicy defines how the message is to be redelivered. You can customize things like
- how many The number of times a message is attempted to be redelivered before it is considered a failure and sent to the dead letter channel.
- the The initial redelivery timeout.
- whether Whether or not exponential backoff is used (, i.e., the time between retries increases using a backoff multiplier).
- whether Whether to use collision avoidance to add some randomness to the timings.
- delay pattern a new option in Camel 2.0, Delay pattern (see below for details).
- Camel 2.11: Whether to allow redelivery during stopping/shutdown.
Once all attempts at redelivering the message fails then the message is forwarded to the dead letter queue.
About
...
Moving Exchange to
...
Dead Letter Queue and Using handled()
handled()
Handled on Dead Letter Channel was introduced in Camel 2.0, this feature does not exist in Camel 1.x
When all attempts of redelivery have failed the Exchange is moved to the dead letter queue (the dead letter endpoint). The exchange is then complete and from the client point of view it was processed. As such the Dead Letter Channel have handled the Exchange.
For instance configuring the dead letter channel as:
...
Using the Fluent Builders
...
Using the Spring XML Extensions
...
The Dead Letter Channel above will clear the caused exception when setException(null)
, by moving the caused exception to a property on the Exchange, with the key Exchange.EXCEPTION_CAUGHT
. Then the Exchange is moved to the jms:queue:dead
destination and the client will not notice the failure.
By default handled is true
.
How to let the client notice the error?
If you want to move the message to the dead letter queue and also let the client notice the error, then you can configure the Dead Letter Channel to not handle the error. For example:
...
errorHandler(deadLetterChannel("jms:queue:dead").maximumRedeliveries(3).redeliverDealy(5000).handled(false));
About Moving Exchange to Dead Letter Queue and Using the Original Message
When all attempts of redelivery have failed the Exchange is moved to the dead letter queue (the dead letter endpoint). As the Dead Letter Channel
is configured to not handle it, it will mark the Exchange as failed so the client will be notified of this error.
Tip | ||
---|---|---|
| ||
See also Exception Clause for more details on the handled policy as this feature was first introduced here and thus we have more docuemntation and samples there. |
About moving Exchange to dead letter queue and using the original message
Available as of Camel 2.0
The option useOriginalMessage
is used for routing the original input message instead of the current message that potentially is modified during routing.
For instance if you have this route:
...
The route listen for JMS messages and validates, transforms and handle it. During this the Exchange payload is transformed/modified. So in case something goes wrong and we want to move the message to another JMS destination, then we can configure our Dead Letter Channel with the useOriginalBody the useOriginalMessage
option. But when we move the Exchange to this destination we do not know in which state the message is in. Did the error happen in before the the transformOrder
or after? So to be sure we want to move the original input message we received from jms:queue:order:input
. So we can do this by enabling the the useOriginalMessage
option as shown below:
...
...
Then the messages routed to the jms:queue:dead
is the original input. If we want to manually retry we can move the JMS message from the failed to the input queue, with no problem as the message is the same as the original we received.
OnRedelivery
...
When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered. See below for sample.
...
...
...
We also support for per onException to set
...
an onRedeliver
. That means you can do special on redelivery for different exceptions, as opposed
...
to onRedelivery
set on Dead Letter Channel can be viewed as a global scope.
Redelivery
...
Default Values
Redelivery In Camel 2.0 redelivery is disabled by default, as opposed to Camel 1.x in which Dead Letter Channel is configured with maximumRedeliveries=5
.
The default redeliver policy will use the following values:
maximumRedeliveries=0
(in Camel 1.x the default value is 5)redeliverDelay=1000L
(1 second, new as of Camel 2.0)- use initialRedeliveryDelay for previous versions
- And the exponential backoff and collision avoidance is turned off
backOffMultiplier
anduseExponentialBackOff
are ignored. - The
retriesExhaustedLogLevel=LoggingLevel.ERROR
retryAttemptedLogLevel=
and retryAttemptedLogLevel are set toLoggingLevel.DEBUG
- Stack traces is not loggedStack traces is now logged are logged for exhausted messages, from Camel 2.2 onwards.
- Handled exceptions are not logged, from Camel 2.3.
logExhaustedMessageHistory
is true for default error handler, and false for dead letter channel.logExhaustedMessageBody
Camel 2.17: is disabled by default to avoid logging sensitive message body/header details. If this option istrue
, thenlogExhaustedMessageHistory
must also betrue
.
maximumRedeliveryDelay = 60 * 1000L
(60 seconds)The maximum redeliver delay ensures that a delay is never longer than the value, default 1 minute. This can happen if you turn on the exponential backoff.when useExponentialBackOff=true
.
The maximumRedeliveries
The maximum redeliveries is the number of re-delivery attempts. By default Camel will try to process the exchange 1 + 5 times. 1 time for the normal attempt and then 5 attempts as redeliveries.
Setting the maximumRedeliveries to a negative value such as the maximumRedeliveries=-1
(or < -1
) will then always redelivery (unlimited).
Setting the the maximumRedeliveries
to =0
will disable any re-delivery attempt.
Camel will log delivery failures at the the DEBUG
logging level by default. You can change this by specifying specifying retriesExhaustedLogLevel
and/or retryAttemptedLogLevel
. See ExceptionBuilderWithRetryLoggingLevelSetTest for an example.
In Camel 2.0 you You can turn logging of stack traces on/off. If turned off Camel will still log the redelivery attempt. Its It's just much less verbose.
Redeliver Delay Pattern
Available as of Camel 2.0
Delay pattern is used as a single option to set a range pattern for delays. If used then When a delay pattern is in use the following options does not no longer apply: (
delay
...
backOffMultiplier
...
useExponentialBackOff
...
useCollisionAvoidance
...
maximumRedeliveryDelay
...
The idea is to set groups of ranges using the following syntax: limit:delay;limit 2:delay 2;limit 3:delay 3;...;limit N:delay N
Each group has two values separated with colon:
limit
= upper limitdelay
= delay in
...
- milliseconds
And the groups is again separated with semi-colon.
...
- The rule of thumb is that the next groups should have a higher limit than the previous group.
Lets clarify this with an example:
delayPattern=5:1000;10:5000;20:20000
That gives us 3 three groups:
5:1000
10:5000
20:20000
Resulting in these delays for between redelivery attemptattempts:
...
- Redelivery attempt number
1..4 =
...
0ms
(as the first group start with 5)
...
- Redelivery attempt number
5..9 =
...
1000ms
(the first group)
...
- Redelivery attempt number
10..19 =
...
5000ms
(the second group)
...
- Redelivery attempt number
20.. =
...
20000ms
(the last group)
Note: The first redelivery attempt is 1
, so the first group should start with 1
or higher.
You can start a group with limit 0 to eg limit 1
to e.g., have a starting delay: delayPattern=
01:1000;5:5000
...
- Redelivery attempt number
1..4 =
...
1000ms
(the first group)
...
- Redelivery attempt number
5.. =
...
5000ms
(the last group)
There is no requirement that the next delay should be higher than the previous. You can use any delay value you like. For example with delayPattern=
01:5000;3:1000
we start with 5 sec delay and then later reduce that to to 1
second.
Redelivery header
When a message is redelivered the DeadLetterChannel will append a customizable header to the message to indicate how many times its been redelivered.
In Before Camel 12.x6: The header is org.apache.camel.redeliveryCount.
In is CamelRedeliveryCounter
, which is also defined on the Exchange.REDELIVERY_COUNTER
.
From Camel 2.06: The header is CamelRedeliveryCounterheader CamelRedeliveryMaxCounter
, which is also defined on the Exchange.REDELIVERY_MAX_COUNTER
, contains the maximum redelivery setting. This header is absent if you use retryWhile
or have unlimited maximum redelivery configured.
And a boolean flag whether it is being redelivered or not (first attempt)
In Camel 1. x: The header org.apache.camel.Redelivered The header CamelRedelivered
contains a boolean if the message is redelivered or not, which is also defined on the Exchange.REDELIVERED
.
Dynamically Calculated Delay From the Exchange
In Camel 2.09 and 2.8.2: The header CamelRedelivered contains a boolean if the message is redelivered or notis CamelRedeliveryDelay
, which is also defined on the Exchange
.REDELIVERED.REDELIVERY_DELAY
. If this header is absent, normal redelivery rules apply.
Which
...
Endpoint Failed
Available as of Camel 2.1
When Camel routes messages it will decorate the Exchange with a property that contains the last endpoint Camel send the Exchange to:
...
The Exchange.TO_ENDPOINT
have the constant value CamelToEndpoint
. This information is updated when Camel sends a message to any endpoint. So if it exists its the last endpoint which Camel send the Exchange to.
When for example processing the Exchange at a given Endpoint and the message is to be moved into the dead letter queue, then Camel also decorates the Exchange with another property that contains that last endpoint:
...
The Exchange.FAILURE_ENDPOINT
have the constant value CamelFailureEndpoint
.
This allows for example you to fetch this information in your dead letter queue and use that for error reporting.
This is useable usable if the Camel route is a bit dynamic such as the dynamic Recipient List so you know which endpoints failed.
NoticeNote: These this information is kept retained on the Exchange even if the message was is subsequently processed successfully processed by a given endpoint , and then later fails only to fail, for example, in a local Bean processing instead. So, beware that this is a hint that helps pinpoint errors.
...
Now suppose the route above and a failure happens in the foo
bean. Then the Exchange.TO_ENDPOINT
and Exchange.FAILURE_ENDPOINT
will still contain the value of http://someserver/somepath
..
OnPrepareFailure
Available as of Camel 2.16
Before the exchange is sent to the dead letter queue, you can use onPrepare
to allow a custom Processor
to prepare the exchange, such as adding information why the Exchange failed.
For example, the following processor adds a header with the exception message:
...
Then configure the error handler to use the processor as follows:
...
Configuring this from XML DSL is as follows:
...
The onPrepare
is also available using the default error handler.
Which Route Failed
Available as of Camel 2.10.4/2.11
When Camel error handler handles an error such as Dead Letter Channel or using Exception Clause with handled=true
, then Camel will decorate the Exchange with the route id where the error occurred.
Example:
...
The Exchange.FAILURE_ROUTE_ID
have the constant value CamelFailureRouteId
. This allows for example you to fetch this information in your dead letter queue and use that for error reporting.
Control if Redelivery is Allowed During Stopping/Shutdown
Available as of Camel 2.11
Before Camel 2.10, Camel would perform redelivery while stopping a route, or shutting down Camel. This has improved a bit in Camel 2.10: Camel will no longer perform redelivery attempts when shutting down aggressively, e.g., during Graceful Shutdown and timeout hit.
From Camel 2.11: there is a new option allowRedeliveryWhileStopping
which you can use to control if redelivery is allowed or not; notice that any in progress redelivery will still be executed. This option can only disallow any redelivery to be executed after the stopping of a route/shutdown of Camel has been triggered. If a redelivery is disallowed then a RejectedExcutionException
is set on the Exchange and the processing of the Exchange stops. This means any consumer will see the Exchange as failed due the RejectedExcutionException
. The default value is true
for backward compatibility.
For example, the following snippet shows how to do this with Java DSL and XML DSL:
Samples
The following example shows how to configure the Dead Letter Channel configuration using the DSL
...
...
You can also configure the RedeliveryPolicy as this example shows
...
How
...
Can I
...
Modify the Exchange
...
Before Redelivery?
In Camel 1.6.0 we added We support directly in Dead Letter Channel to set a Processor that is executed before each redelivery attempt. When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered. Here we configure the Dead Letter Channel to use our processor MyRedeliveryProcessor
to be executed before each redelivery.
...
...
And this is the processor MyRedeliveryProcessor
where we alter the message.
...
...
How Can I Log What Caused the Dead Letter Channel to be Invoked?
You often need to know what went wrong that caused the Dead Letter Channel to be used and it does not offer logging for this purpose. So the Dead Letter Channel's endpoint can be set to a endpoint of our own (such as direct:deadLetterChannel
). We write a route to accept this Exchange and log the Exception, then forward on to where we want the failed Exchange moved to (which might be a DLQ queue for instance). See also http://stackoverflow.com/questions/13711462/logging-camel-exceptions-and-sending-to-the-dead-letter-channel
...