Page History

...

Warning

title	Avoid reading files currently being written by another application

Beware the JDK File IO API is a bit limited in detecting whether another application is currently writing/copying a file. And the implementation can be different depending on OS platform as well. This could lead to that Camel thinks the file is not locked by another process and start consuming it. Therefore you have to do you own investigation what suites your environment. To help with this Camel provides different readLock options and doneFileName option that you can use. See also the section Consuming files from folders where others drop files directly.

URI Options

Common

Div

class	confluenceTableSmall

Name	Default Value	Description
`autoCreate`	`true`	Automatically create missing directories in the file's pathname. For the file consumer, that means creating the starting directory. For the file producer, it means the directory the files should be written to.
`bufferSize`	128kb	Write buffer sized in bytes.
`fileName`	`null`	Use Expression such as File Language to dynamically set the filename. For consumers, it's used as a filename filter. For producers, it's used to evaluate the filename to write. If an expression is set, it take precedence over the `CamelFileName` header. (Note: The header itself can also be an Expression). The expression options support both `String` and `Expression` types. If the expression is a `String` type, it is always evaluated using the File Language. If the expression is an `Expression` type, the specified `Expression` type is used - this allows you, for instance, to use OGNL expressions. For the consumer, you can use it to filter filenames, so you can for instance consume today's file using the File Language syntax: `mydata-$`

Wiki Markup

{div:class=confluenceTableSmall} || Name || Default Value || Description || | {{autoCreate}} | {{true}} | Automatically create missing directories in the file's pathname. For the file consumer, that means creating the starting directory. For the file producer, it means the directory the files should be written to. | | {{bufferSize}} | 128kb | Write buffer sized in bytes. | | {{fileName}} | {{null}} | Use [Expression] such as [File Language] to dynamically set the filename. For consumers, it's used as a filename filter. For producers, it's used to evaluate the filename to write. If an expression is set, it take precedence over the {{CamelFileName}} header. (*Note:* The header itself can also be an [Expression]). The expression options support both {{String}} and {{Expression}} types. If the expression is a {{String}} type, it is *always* evaluated using the [File Language]. If the expression is an {{Expression}} type, the specified {{Expression}} type is used - this allows you, for instance, to use [OGNL] expressions. For the consumer, you can use it to filter filenames, so you can for instance consume today's file using the [File Language] syntax: {{mydata-$\

{date:now:yyyyMMdd}.txt

}}

.

From

*

Camel

2.11

*

onwards

the

producers

support

the

{{

CamelOverruleFileName

}}

header

which

takes

precedence

over

any

existing

{{

CamelFileName

}}

header;

the

{{

CamelOverruleFileName

}}

is

a

header

that

is

used

only

once,

and

makes

it

easier

as

this

avoids

to

temporary

store

{{

CamelFileName

}}

and

have

to

restore

it

afterwards.

| | {{flatten}} | {{false}} | Flatten is used to flatten the file name path to strip any leading paths, so it's just the file name. This allows you to consume recursively into sub-directories, but when you eg write the files to another directory they will be written in a single directory. Setting this to {{true}} on the producer enforces that any file name recived in {{CamelFileName}} header will be stripped for any leading paths. | | {{charset}} | {{null}} | *Camel


`flatten`	`false`	Flatten is used to flatten the file name path to strip any leading paths, so it's just the file name. This allows you to consume recursively into sub-directories, but when you eg write the files to another directory they will be written in a single directory. Setting this to `true` on the producer enforces that any file name recived in `CamelFileName` header will be stripped for any leading paths.
`charset`	`null`	Camel 2.9.3:

*

this

option

is

used

to

specify

the

encoding

of

the

file,

and

camel

will

set

the

Exchange

property

with

{{

Exchange.CHARSET_NAME

}}

with

the

value

of

this

option.

You

can

use

this

on

the

consumer,

to

specify

the

encodings

of

the

files,

which

allow

Camel

to

know

the

charset

it

should

load

the

file

content

in

case

the

file

content

is

being

accessed.

Likewise

when

writing

a

file,

you

can

use

this

option

to

specify

which

charset

to

write

the

file

as

well.

See

further

below

for

a

examples

and

more

important

details.

| | {{copyAndDeleteOnRenameFail}} | {{true}} | *Camel


`copyAndDeleteOnRenameFail`	`true`	Camel 2.9

*

:

whether

to

fallback

and

do

a

copy

and

delete

file,

in

case

the

file

could

not

be

renamed

directly.

This

option

is

not

available

for

the

[

FTP

|FTP2]

component.

| | {{renameUsingCopy}} | {{false}} | *Camel


`renameUsingCopy`	`false`	Camel 2.13.1

*

:

Perform

rename

operations

using

a

copy

and

delete

strategy.

This

is

primarily

used

in

environments

where

the

regular

rename

operation

is

unreliable

(e.g.

across

different

file

systems

or

networks).

This

option

takes

precedence

over

the

{{

copyAndDeleteOnRenameFail

}}

parameter

that

will

automatically

fall

back

to

the

copy

and

delete

strategy,

but

only

after

additional

delays.

{div}

Consumer

Div

class	confluenceTableSmall

Wiki Markup

{div:class=confluenceTableSmall} || Name || Default Value || Description || | {{initialDelay}} | {{1000}} | Milliseconds before polling the file/directory starts. | | {{delay}} | {{500}} | Milliseconds before the next poll of the file/directory. | | {{useFixedDelay}} | | Controls if fixed delay or fixed rate is used. See [ScheduledExecutorService|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html] in JDK for details. In *Camel 2.7.x* or older the default value is {{false}}. From *Camel 2.8* onwards the default value is {{true}}. | | {{runLoggingLevel}} | {{TRACE}} | *Camel 2.8:* The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. | | {{recursive}} | {{false}} | If a directory, will look for files in all the sub-directories as well. | | {{delete}} | {{false}} | If {{true}}, the file will be deleted *after* it is processed successfully. | | {{noop}} | {{false}} | If {{true}}, the file is not moved or deleted in any way. This option is good for readonly data, or for [ETL] type requirements. If {{noop=true}}, Camel will set {{idempotent=true}} as well, to avoid consuming the same files over and over again. | | {{preMove}} | {{null}} | [Expression] (such as [File Language]) used to dynamically set the filename when moving it *before* processing. For example to move in-progress files into the {{order}} directory set this value to {{order}}. | | {{move}} | {{.camel}} | [Expression] (such as [File Language]) used to dynamically set the filename when moving it *after* processing. To move files into a {{.done}} subdirectory just enter {{.done}}. | | {{moveFailed}} | {{null}} | [Expression] (such as [File Language]) used to dynamically set a different target directory when moving files _in case of_ processing (configured via {{move}} defined above) failed. For example, to move files into a {{.error}} subdirectory use: {{.error}}. Note: When moving the files to the “fail” location Camel will *handle* the error and will not pick up the file again. | | {{include}} | {{null}} | Is used to include files, if filename matches the regex pattern. | | {{exclude}} | {{null}} | Is used to exclude files, if filename matches the regex pattern. | | {{antInclude}} | {{null}} | *Camel 2.10:* Ant style filter inclusion, for example {{antInclude=\*}}{{{}*/*{}}}{{.txt}}. Multiple inclusions may be specified in comma-delimited format. See [below|#Filtering using ANT path matcher] for more details about ant path filters. | | {{antExclude}} | {{null}} | *Camel 2.10:* Ant style filter exclusion. If both {{antInclude}} and {{antExclude}} are used, {{antExclude}} takes precedence over {{antInclude}}. Multiple exclusions may be specified in comma-delimited format. See [below|#Filtering using ANT path matcher] for more details about ant path filters. | | {{antFilterCaseSensitive}} | {{true}} | *Camel 2.11:* Ant style filter which is case sensitive or not. | | {{idempotent}} | {{false}} | Option to use the [Idempotent Consumer] EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If {{noop=true}} then idempotent will be enabled as well to avoid consuming the same files over and over again. | | {{idempotentKey}} | {{Expression}} | *Camel 2.11:* To use a custom idempotent key. By default the absolute path of the file is used. You can use the [File Language], for example to use the file name and file size, you can do: {code}

Name

Default Value

Description

initialDelay

1000

Milliseconds before polling the file/directory starts.

delay

500

Milliseconds before the next poll of the file/directory.

useFixedDelay

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. In Camel 2.7.x or older the default value is false. From Camel 2.8 onwards the default value is true.

runLoggingLevel

TRACE

Camel 2.8: The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

recursive

false

If a directory, will look for files in all the sub-directories as well.

delete

false

If true, the file will be deleted after it is processed successfully.

noop

false

If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, Camel will set idempotent=true as well, to avoid consuming the same files over and over again.

preMove

null

Expression (such as File Language) used to dynamically set the filename when moving it before processing. For example to move in-progress files into the order directory set this value to order.

move

.camel

Expression (such as File Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done.

moveFailed

null

Expression (such as File Language) used to dynamically set a different target directory when moving files in case of processing (configured via move defined above) failed. For example, to move files into a .error subdirectory use: .error. Note: When moving the files to the “fail” location Camel will handle the error and will not pick up the file again.

include

null

Is used to include files, if filename matches the regex pattern.

exclude

null

Is used to exclude files, if filename matches the regex pattern.

antInclude

null

Camel 2.10: Ant style filter inclusion, for example antInclude=*/.txt. Multiple inclusions may be specified in comma-delimited format. See below for more details about ant path filters.

antExclude

null

Camel 2.10: Ant style filter exclusion. If both antInclude and antExclude are used, antExclude takes precedence over antInclude. Multiple exclusions may be specified in comma-delimited format. See below for more details about ant path filters.

antFilterCaseSensitive

true

Camel 2.11: Ant style filter which is case sensitive or not.

idempotent

false

Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.

idempotentKey

Expression

Camel 2.11: To use a custom idempotent key. By default the absolute path of the file is used. You can use the File Language, for example to use the file name and file size, you can do:

Code Block

idempotentKey=${file:name}-${file:size}

{code}. | | {{idempotentRepository}} | {{null}} | A pluggable repository [

.

idempotentRepository

null

A pluggable repository org.apache.camel.spi

.IdempotentRepository|http://camel.apache

.

org/maven/current/camel-core/apidocs/org/apache/camel/spi/

IdempotentRepository

.html]

which

by

default

use

{{

MemoryMessageIdRepository

}}

if

none

is

specified

and

{{

idempotent

}}

is

{{

true

}}. | | {{inProgressRepository}} | {{memory}} | A pluggable

.
`inProgressRepository`	`memory`	A pluggable in-progress

repository

[

org.apache.camel.spi.

IdempotentRepository|http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/

IdempotentRepository

.html]

.

The

in-progress

repository

is

used

to

account

the

current

in

progress

files

being

consumed.

By

default

a

memory

based

repository

is

used.

| | {{filter}} | {{null}} | Pluggable filter as a {{


`filter`	`null`	Pluggable filter as a `org.apache.camel.component.file.GenericFileFilter`

}}

class.

Will

skip

files

if

filter

returns

{{

false

}}

in

its

{{

accept()

}}

method.

More

details

in

section

below.

| | {{sorter}} | {{null}} | Pluggable sorter as a [


`sorter`	`null`	Pluggable sorter as a java.util.Comparator<org.apache.camel.component.file.GenericFile>

|http://java.sun.com/j2se/1.5.0/docs/api/java/util/Comparator.html] class. | | {{sortBy}} | {{null}} | Built-in sort using the [File Language]. Supports nested sorts, so you can have a sort by file name and as a 2nd group sort by modified date. See sorting section below for details. | | {{readLock}} | {{markerFile}} | Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. \\ This option provides the build in strategies: \\ {{markerFile}} Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is *not* available for the [FTP|FTP2] component. \\ {{changed}} is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec. to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option {{readLockCheckInterval}} can be used to set the check frequency. This option is *only* avail for the [FTP|FTP2] component from *Camel 2.8* onwards. Notice that from *Camel 2.10.1* onwards the [FTP|FTP2] option {{fastExistsCheck}} can be enabled to speedup this readLock strategy, if the FTP server support the LIST operation with a full file name (some servers may not). \\ {{fileLock}} is for using {{java.nio.channels.FileLock}}. This option is *not* avail for the [FTP|FTP2] component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks. \\ {{rename}} is for using a try to rename the file as a test if we can get exclusive read-lock. \\ {{none}} is for no read locks at all. \\ Notice from *Camel 2.10* onwards the read locks {{changed}}, {{fileLock}} and {{rename}} will also use a {{markerFile}} as well, to ensure not picking up files that may be in process by another Camel consumer running on another node (eg cluster). This is only supported by the file component (not the ftp component). | | {{readLockTimeout}} | {{10000}} | Optional timeout in millis for the read-lock, if supported by the read-lock. If the read-lock could not be granted and the timeout triggered, then Camel will skip the file. At next poll Camel, will try the file again, and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. In *Camel 2.0* the default value is 0. Starting with *Camel 2.1* the default value is 10000. Currently {{fileLock}}, {{changed}} and {{rename}} support the timeout. *Notice:* For [FTP|FTP2] the default {{readLockTimeout}} value is {{20000}} instead of {{10000}}. | | {{readLockCheckInterval}} | {{1000}} | *Camel 2.6:* Interval in millis for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the {{changed}} read lock, you can set a higher interval period to cater for _slow writes_. The default of 1 sec. may be _too fast_ if the producer is very slow writing the file. For [FTP|FTP2] the default {{readLockCheckInterval}} is {{5000}}. | | {{readLockMinLength}} | {{1}} | *Camel 2.10.1:* This option applied only for {{readLock=changed}}. This option allows you to configure a minimum file length. By default Camel expects the file to contain data, and thus the default value is 1. You can set this option to zero, to allow consuming zero-length files. | | {{readLockLoggingLevel}} | {{WARN}} | *Camel 2.12:* Logging level used when a read lock could not be acquired. By default a WARN is logged. You can change this level, for example to OFF to not have any logging. This option is only applicable for readLock of types: changed, fileLock, rename. | | {{readLockMarkerFile}} | {{true}} | *Camel 2.14:* Whether to use marker file with the {{changed}}, {{rename}}, or {{exclusive}} read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to {{false}}. For example if you do not want to write marker files to the file systems by the Camel application. | | {{directoryMustExist}} | {{false}} | *Camel 2.5:* Similar to {{startingDirectoryMustExist}} but this applies during polling recursive sub directories. | | {{doneFileName}} | {{null}} | *Camel 2.6:* If provided, Camel will only consume files if a _done_ file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The _done_ file is *always* expected in the same folder as the original file. See _using done file_ and _writing done file_ sections for examples. | | {{exclusiveReadLockStrategy}} | {{null}} | Pluggable read-lock as a {{

class.
`sortBy`	`null`	Built-in sort using the File Language. Supports nested sorts, so you can have a sort by file name and as a 2nd group sort by modified date. See sorting section below for details.
`readLock`	`markerFile`	Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies: `markerFile` Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component. `changed` is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec. to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option `readLockCheckInterval` can be used to set the check frequency. This option is only avail for the FTP component from Camel 2.8 onwards. Notice that from Camel 2.10.1 onwards the FTP option `fastExistsCheck` can be enabled to speedup this readLock strategy, if the FTP server support the LIST operation with a full file name (some servers may not). `fileLock` is for using `java.nio.channels.FileLock`. This option is not avail for the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks. `rename` is for using a try to rename the file as a test if we can get exclusive read-lock. `none` is for no read locks at all. Notice from Camel 2.10 onwards the read locks `changed`, `fileLock` and `rename` will also use a `markerFile` as well, to ensure not picking up files that may be in process by another Camel consumer running on another node (eg cluster). This is only supported by the file component (not the ftp component).
`readLockTimeout`	`10000`	Optional timeout in millis for the read-lock, if supported by the read-lock. If the read-lock could not be granted and the timeout triggered, then Camel will skip the file. At next poll Camel, will try the file again, and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. In Camel 2.0 the default value is 0. Starting with Camel 2.1 the default value is 10000. Currently `fileLock`, `changed` and `rename` support the timeout. Notice: For FTP the default `readLockTimeout` value is `20000` instead of `10000`.
`readLockCheckInterval`	`1000`	Camel 2.6: Interval in millis for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the `changed` read lock, you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. For FTP the default `readLockCheckInterval` is `5000`.
`readLockMinLength`	`1`	Camel 2.10.1: This option applied only for `readLock=changed`. This option allows you to configure a minimum file length. By default Camel expects the file to contain data, and thus the default value is 1. You can set this option to zero, to allow consuming zero-length files.
`readLockLoggingLevel`	`WARN`	Camel 2.12: Logging level used when a read lock could not be acquired. By default a WARN is logged. You can change this level, for example to OFF to not have any logging. This option is only applicable for readLock of types: changed, fileLock, rename.
`readLockMarkerFile`	`true`	Camel 2.14: Whether to use marker file with the `changed`, `rename`, or `exclusive` read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to `false`. For example if you do not want to write marker files to the file systems by the Camel application.
`directoryMustExist`	`false`	Camel 2.5: Similar to `startingDirectoryMustExist` but this applies during polling recursive sub directories.
`doneFileName`	`null`	Camel 2.6: If provided, Camel will only consume files if a done file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The done file is always expected in the same folder as the original file. See using done file and writing done file sections for examples.
`exclusiveReadLockStrategy`	`null`	Pluggable read-lock as a `org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy`

}}

implementation.

| | {{maxMessagesPerPoll}} | {{0}} | An integer to define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it. See more details at [Batch Consumer]. *Notice:* If this option is in use then the [File|File2] and [FTP|FTP2] components will limit *before* any sorting. For example if you have 100000 files and use {{maxMessagesPerPoll=500}}, then only the first 500 files will be picked up, and then sorted. You can use the {{eagerMaxMessagesPerPoll}} option and set this to {{false}} to allow to scan all files first and then sort afterwards. | | {{eagerMaxMessagesPerPoll}} | {{true}} | *Camel 2.9.3:* Allows for controlling whether the limit from {{maxMessagesPerPoll}} is eager or not. If eager then the limit is during the scanning of files. Where as {{false}} would scan all files, and then perform sorting. Setting this option to {{false}} allows for sorting all files first, and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting. | | {{minDepth}} | 0 | *Camel 2.8*: The minimum depth to start processing when recursively processing a directory. Using {{minDepth=1}} means the base directory. Using {{minDepth=2}} means the first sub directory. This option is supported by [FTP|FTP2] consumer from *Camel 2.8.2, 2.9* onwards. | | {{maxDepth}} | {{Integer.MAX_VALUE}} | *Camel 2.8:* The maximum depth to traverse when recursively processing a directory. This option is supported by [FTP|FTP2] consumer from *Camel 2.8.2, 2.9* onwards. | | {{processStrategy}} | {{null}} | A pluggable {{


`maxMessagesPerPoll`	`0`	An integer to define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it. See more details at Batch Consumer. Notice: If this option is in use then the File and FTP components will limit before any sorting. For example if you have 100000 files and use `maxMessagesPerPoll=500`, then only the first 500 files will be picked up, and then sorted. You can use the `eagerMaxMessagesPerPoll` option and set this to `false` to allow to scan all files first and then sort afterwards.
`eagerMaxMessagesPerPoll`	`true`	Camel 2.9.3: Allows for controlling whether the limit from `maxMessagesPerPoll` is eager or not. If eager then the limit is during the scanning of files. Where as `false` would scan all files, and then perform sorting. Setting this option to `false` allows for sorting all files first, and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting.
`minDepth`	0	Camel 2.8: The minimum depth to start processing when recursively processing a directory. Using `minDepth=1` means the base directory. Using `minDepth=2` means the first sub directory. This option is supported by FTP consumer from Camel 2.8.2, 2.9 onwards.
`maxDepth`	`Integer.MAX_VALUE`	Camel 2.8: The maximum depth to traverse when recursively processing a directory. This option is supported by FTP consumer from Camel 2.8.2, 2.9 onwards.
`processStrategy`	`null`	A pluggable `org.apache.camel.component.file.GenericFileProcessStrategy`

}}

allowing

you

to

implement

your

own

{{

readLock

}}

option

or

similar.

Can

also

be

used

when

special

conditions

must

be

met

before

a

file

can

be

consumed,

such

as

a

special

_

ready

_

file

exists.

If

this

option

is

set

then

the

{{

readLock

}}

option

does

not

apply.

| | {{startingDirectoryMustExist}} | {{false}} | *Camel


`startingDirectoryMustExist`	`false`	Camel 2.5:

*

Whether

the

starting

directory

must

exist.

Mind

that

the

{{

autoCreate

}}

option

is

default

enabled,

which

means

the

starting

directory

is

normally

auto

created

if

it

doesn't

exist.

You

can

disable

{{

autoCreate

}}

and

enable

this

to

ensure

the

starting

directory

must

exist.

Will

thrown

an

exception

if

the

directory

doesn't

exist.

| | {{pollStrategy}} | {{null}} | A pluggable {{


`pollStrategy`	`null`	A pluggable `org.apache.camel.spi.PollingConsumerPollStrategy`

}}

allowing

you

to

provide

your

custom

implementation

to

control

error

handling

usually

occurred

during

the

{{

poll

}}

operation

*

before

*

an

[

Exchange

]

have

been

created

and

being

routed

in

Camel.

In

other

words

the

error

occurred

while

the

polling

was

gathering

information,

for

instance

access

to

a

file

network

failed

so

Camel

cannot

access

it

to

scan

for

files.

The

default

implementation

will

log

the

caused

exception

at

{{

WARN

}}

level

and

ignore

it.

| | {{sendEmptyMessageWhenIdle}} | {{false}} | *Camel


`sendEmptyMessageWhenIdle`	`false`	Camel 2.9:

*

If

the

polling

consumer

did

not

poll

any

files,

you

can

enable

this

option

to

send

an

empty

message

(no

body)

instead. | | {{

instead.

consumer.bridgeErrorHandler

}} | {{false}} | *Camel

false

Camel 2.10:

*

Allows

for

bridging

the

consumer

to

the

Camel

routing

[

Error

Handler

]

,

which

mean

any

exceptions

occurred

while

trying

to

pickup

files,

or

the

likes,

will

now

be

processed

as

a

message

and

handled

by

the

routing

[

Error

Handler

]

.

By

default

the

consumer

will

use

the

{{

org.apache.camel.spi.ExceptionHandler

}}

to

deal

with

exceptions,

that

by

default

will

be

logged

at

WARN/ERROR

level

and

ignored.

See

further

below

on

this

page

fore

more

details,

at

section

_

How

to

use

the

Camel

error

handler

to

deal

with

exceptions

triggered

outside

the

routing

engine

_. | | {{scheduledExecutorService}} | {{null}} | *Camel

.
`scheduledExecutorService`	`null`	Camel 2.10:

*

Allows

for

configuring

a

custom/shared

thread

pool

to

use

for

the

consumer.

By

default

each

consumer

has

its

own

single

threaded

thread

pool.

This

option

allows

you

to

share

a

thread

pool

among

multiple

file

consumers.

| | {{scheduler}} | {{null}} | *Camel


`scheduler`	`null`	Camel 2.12:

*

To

use

a

custom

scheduler

to

trigger

the

consumer

to

run.

See

more

details

at

[

Polling

Consumer

]

,

for

example

there

is

a

[

Quartz2

]

,

and

[

Spring

]

based

scheduler

that

supports

CRON

expressions. | | {{backoffMultiplier}} | {{0}} | *Camel

expressions.
`backoffMultiplier`	`0`	Camel 2.12:

*

To

let

the

scheduled

polling

consumer

backoff

if

there

has

been

a

number

of

subsequent

idles/errors

in

a

row.

The

multiplier

is

then

the

number

of

polls

that

will

be

skipped

before

the

attempt

is

happening

again.

When

this

option

is

in

use

then

{{

backoffIdleThreshold

}}

and/or

{{

backoffErrorThreshold

}}

must

also

be

configured.

See

more

details

at

[

Polling

Consumer

]. | | {{backoffIdleThreshold}} | {{0}} | *Camel

.
`backoffIdleThreshold`	`0`	Camel 2.12:

*

The

number

of

subsequent

idle

polls

that

should

happen

before

the

{{

backoffMultipler

}}

should

kick-in.

| | {{backoffErrorThreshold}} | {{0}} | *Camel


`backoffErrorThreshold`	`0`	Camel 2.12:

*

The

number

of

subsequent

error

polls

(failed

due

some

error)

that

should

happen

before

the

{{

backoffMultipler

}}

should

kick-in.

| {div}

Default behavior for file consumer

By default the file is locked for the duration of the processing.
After the route has completed, files are moved into the .camel subdirectory, so that they appear to be deleted.
The File Consumer will always skip any file whose name starts with a dot, such as ., .camel, .m2 or .groovy.
Only files (not directories) are matched for valid filename, if options such as: include or exclude are used.

Producer

Div

class	confluenceTableSmall

Name	Default Value	Description
`fileExist`	`Override`	What to do if a file already exists with the same name. The following values can be specified: Override, Append, Fail, Ignore, Move, and TryRename (Camel

Wiki Markup

{div:class=confluenceTableSmall} || Name || Default Value || Description || | {{fileExist}} | {{Override}} | What to do if a file already exists with the same name. The following values can be specified: *Override*, *Append*, *Fail*, *Ignore*, *Move*, and *TryRename* (Camel

2.11.1).

{{

Override

}}

,

which

is

the

default,

replaces

the

existing

file.

{{

Append

}}

adds

content

to

the

existing

file.

{{

Fail

}}

throws

a

{{

GenericFileOperationException

}}

,

indicating

that

there

is

already

an

existing

file.

{{

Ignore

}}

silently

ignores

the

problem

and

*

does

not

*

override

the

existing

file,

but

assumes

everything

is

okay.

The

{{

Move

}}

option

requires

*

Camel

2.10.1

*

onwards,

and

the

corresponding

{{

moveExisting

}}

option

to

be

configured

as

well.

The

option

{{

eagerDeleteTargetFile

}}

can

be

used

to

control

what

to

do

if

an

moving

the

file,

and

there

exists

already

an

existing

file,

otherwise

causing

the

move

operation

to

fail.

The

{{

Move

}}

option

will

move

any

existing

files,

before

writing

the

target

file.

{{

TryRename

}} *

Camel

2.11.1

*

is

only

applicable

if

{{

tempFileName

}}

option

is

in

use.

This

allows

to

try

renaming

the

file

from

the

temporary

name

to

the

actual

name,

without

doing

any

exists

check.

This

check

may

be

faster

on

some

file

systems

and

especially

FTP

servers.

| | {{tempPrefix}} | {{null}} | This option is used to write the file using a temporary name and then, after the write is complete, rename it to the real name. Can be used to identify files being written and also avoid consumers (not using exclusive read locks) reading in progress files. Is often used by [FTP|FTP2] when uploading big files. | | {{tempFileName}} | {{null}} | *Camel 2.1:* The *same* as {{tempPrefix}} option but offering a more fine grained control on the naming of the temporary filename as it uses the [File Language]. | | {{moveExisting}} | {{null}} | *Camel 2.10.1:* [Expression] (such as [File Language]) used to compute file name to use when {{fileExist=Move}} is configured. To move files into a {{backup}} subdirectory just enter {{backup}}. This option only supports the following [File Language] tokens: "file:name",


`tempPrefix`	`null`	This option is used to write the file using a temporary name and then, after the write is complete, rename it to the real name. Can be used to identify files being written and also avoid consumers (not using exclusive read locks) reading in progress files. Is often used by FTP when uploading big files.
`tempFileName`	`null`	Camel 2.1: The same as `tempPrefix` option but offering a more fine grained control on the naming of the temporary filename as it uses the File Language.
`moveExisting`	`null`	Camel 2.10.1: Expression (such as File Language) used to compute file name to use when `fileExist=Move` is configured. To move files into a `backup` subdirectory just enter `backup`. This option only supports the following File Language tokens: "file:name", "file:name.ext",

"file:name.noext",

"file:onlyname",

"file:onlyname.noext",

"file:ext",

and

"file:parent".

Notice

the

"file:parent"

is

not

supported

by

the

[

FTP

|FTP2]

component,

as

the

FTP

component

can

only

move

any

existing

files

to

a

relative

directory

based

on

current

dir

as

base. | | {{keepLastModified}} | {{false}} | *Camel

base.
`keepLastModified`	`false`	Camel 2.2:

*

Will

keep

the

last

modified

timestamp

from

the

source

file

(if

any).

Will

use

the

{{

Exchange.FILE_LAST_MODIFIED

}}

header

to

located

the

timestamp.

This

header

can

contain

either

a

{{

java.util.Date

}}

or

{{

long

}}

with

the

timestamp.

If

the

timestamp

exists

and

the

option

is

enabled

it

will

set

this

timestamp

on

the

written

file.

*

Note:

*

This

option

only

applies

to

the

*

file

*

producer.

You

_

cannot

_

use

this

option

with

any

of

the

ftp

producers.

| | {{eagerDeleteTargetFile}} | {{true}} | *Camel


`eagerDeleteTargetFile`	`true`	Camel 2.3:

*

Whether

or

not

to

eagerly

delete

any

existing

target

file.

This

option

only

applies

when

you

use

{{

fileExists=Override

}}

and

the

{{

tempFileName

}}

option

as

well.

You

can

use

this

to

disable

(set

it

to

false)

deleting

the

target

file

before

the

temp

file

is

written.

For

example

you

may

write

big

files

and

want

the

target

file

to

exists

during

the

temp

file

is

being

written.

This

ensure

the

target

file

is

only

deleted

until

the

very

last

moment,

just

before

the

temp

file

is

being

renamed

to

the

target

filename.

From

*

Camel

2.10.1

*

onwards

this

option

is

also

used

to

control

whether

to

delete

any

existing

files

when

{{

fileExist=Move

}}

is

enabled,

and

an

existing

file

exists.

If

this

option

copyAndDeleteOnRenameFailis

false,

then

an

exception

will

be

thrown

if

an

existing

file

existed,

if

its

true,

then

the

existing

file

is

deleted

before

the

move

operation.

| | {{doneFileName}} | {{null}} | *Camel


`doneFileName`	`null`	Camel 2.6:

*

If

provided,

then

Camel

will

write

a

2nd

_

done

_

file

when

the

original

file

has

been

written.

The

_

done

_

file

will

be

empty.

This

option

configures

what

file

name

to

use.

Either

you

can

specify

a

fixed

name.

Or

you

can

use

dynamic

placeholders.

The

_

done

_

file

will

*

always

*

be

written

in

the

same

folder

as

the

original

file.

See

_

writing

done

file

_

section

for

examples.

| | {{allowNullBody}} | {{false}} | *Camel


`allowNullBody`	`false`	Camel 2.10.1:

*

Used

to

specify

if

a

null

body

is

allowed

during

file

writing.

If

set

to

true

then

an

empty

file

will

be

created,

when

set

to

false,

and

attempting

to

send

a

null

body

to

the

file

component,

a

GenericFileWriteException

of

'Cannot

write

null

body

to

file.'

will

be

thrown.

If

the

`fileExist`

option

is

set

to

'Override',

then

the

file

will

be

truncated,

and

if

set

to

`append`

the

file

will

remain

unchanged.

| | {{forceWrites}} | {{true}} | *Camel


`forceWrites`	`true`	Camel 2.10.5/2.11:

*

Whether

to

force

syncing

writes

to

the

file

system.

You

can

turn

this

off

if

you

do

not

want

this

level

of

guarantee,

for

example

if

writing

to

logs

/

audit

logs

etc;

this

would

yield

better

performance.

| {div}

Default behavior for file producer

...

The following headers are supported by this component:

File producer only

Wiki Markup

{div:class=confluenceTableSmall}
|| Header || Description ||
| {{CamelFileName}} | Specifies the name of the file to write (relative to the endpoint directory). The name can be a {{String}}; a {{String}} with a [File Language] or [Simple] expression; or an [Expression] object. If it's {{null}} then Camel will auto-generate a filename based on the message unique ID. |
| {{CamelFileNameProduced}} | The actual absolute filepath (path + name) for the output file that was written. This header is set by Camel and its purpose is providing end-users with the name of the file that was written. |
| {{CamelOverruleFileName}} | *Camel 2.11:* Is used for overruling {{CamelFileName}} header and use the value instead (but only once, as the producer will remove this header after writing the file). The value can be only be a String. Notice that if the option {{fileName}} has been configured, then this is still being evaluated. |
{div}

File consumer only

Div

class	confluenceTableSmall

Header	Description
`CamelFileName`	Specifies the name of the file to write (relative to the endpoint directory). The name can be a `String`; a `String` with a File Language or Simple expression; or an Expression object. If it's `null` then Camel will auto-generate a filename based on the message unique ID.
`CamelFileNameProduced`	The actual absolute filepath (path + name) for the output file that was written. This header is set by Camel and its purpose is providing end-users with the name of the file that was written.
`CamelOverruleFileName`	Camel 2.11: Is used for overruling `CamelFileName` header and use the value instead (but only once, as the producer will remove this header after writing the file). The value can be only be a String. Notice that if the option `fileName` has been configured, then this is still being evaluated.

File consumer only

Div

class	confluenceTableSmall

Header	Description
`CamelFileName`	Name of the consumed file as a relative file path with offset from the starting directory configured on the endpoint.
`CamelFileNameOnly`	Only the file name (the name with no leading paths).
`CamelFileAbsolute`	A `boolean` option specifying whether the consumed file denotes an absolute path or not. Should normally be `false` for relative paths. Absolute paths should normally not be used but we added to the move option to allow moving files to absolute paths. But can be used elsewhere as well.
`CamelFileAbsolutePath`	The absolute path to the file. For relative files this path holds the relative path instead.
`CamelFilePath`	The file path. For relative files this is the starting directory + the relative filename. For absolute files this is the absolute path.
`CamelFileRelativePath`	The relative path.
`CamelFileParent`	The parent path.
`CamelFileLength`	A `long` value containing the file size.
`CamelFileLastModified`	A `Long` value containing the last modified timestamp of the file. In Camel 2.10.3 and older the type is `Date`.

Wiki Markup

{div:class=confluenceTableSmall} || Header || Description || | {{CamelFileName}} | Name of the consumed file as a relative file path with offset from the starting directory configured on the endpoint. | | {{CamelFileNameOnly}} | Only the file name (the name with no leading paths). | | {{CamelFileAbsolute}} | A {{boolean}} option specifying whether the consumed file denotes an absolute path or not. Should normally be {{false}} for relative paths. Absolute paths should normally not be used but we added to the move option to allow moving files to absolute paths. But can be used elsewhere as well. | | {{CamelFileAbsolutePath}} | The absolute path to the file. For relative files this path holds the relative path instead. | | {{CamelFilePath}} | The file path. For relative files this is the starting directory + the relative filename. For absolute files this is the absolute path. | | {{CamelFileRelativePath}} | The relative path. | | {{CamelFileParent}} | The parent path. | | {{CamelFileLength}} | A {{long}} value containing the file size. | | {{CamelFileLastModified}} | A {{Long}} value containing the last modified timestamp of the file. In *Camel 2.10.3 and older* the type is {{Date}}. | {div}

Batch Consumer

This component implements the Batch Consumer.

...

As the file consumer is BatchConsumer it supports batching the files it polls. By batching it means that Camel will add some properties to the Exchange so you know the number of files polled the current index in that order.

Div

class	confluenceTableSmall

Property	Description
`CamelBatchSize`	The total number of files that was polled in this batch.
`CamelBatchIndex`	The current index of the batch. Starts from 0.
`CamelBatchComplete`	A `boolean` value indicating the last Exchange in the batch. Is only `true` for the last entry.

Wiki Markup

{div:class=confluenceTableSmall} || Property || Description || | {{CamelBatchSize}} | The total number of files that was polled in this batch. | | {{CamelBatchIndex}} | The current index of the batch. Starts from 0. | | {{CamelBatchComplete}} | A {{boolean}} value indicating the last [Exchange] in the batch. Is only {{true}} for the last entry. | {div}

This allows you for instance to know how many files exists in this batch and for instance let the Aggregator2 aggregate this number of files.

...

Child pages

Versions Compared

Old Version 156

New Version 157

Key

URI Options

Common

Consumer

Default behavior for file consumer

Producer

Default behavior for file producer

File producer only

File consumer only

File consumer only

Batch Consumer