Page History

initialDelay

1000

Milliseconds before polling the file/directory starts.

delay

500

Milliseconds before the next poll of the file/directory.

useFixedDelay

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

In Camel 2.7.x or older the default value is false.

From Camel 2.8 onward the default value is true.

runLoggingLevel

TRACE

Camel 2.8: The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

recursive

false

If a directory, will look for files in all the sub-directories as well.

delete

false

If true, the file will be deleted after it is processed successfully.

noop

false

If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, Camel will set idempotent=true as well, to avoid consuming the same files over and over again.

preMove

null

Expression (such as File Language) used to dynamically set the filename when moving it before processing. For example to move in-progress files into the order directory set this value to order.

move

.camel

Expression (such as File Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done.

moveFailed

null

Expression (such as File Language) used to dynamically set a different target directory when moving files in case of processing (configured via move defined above) failed.

For example, to move files into a .error subdirectory use: .error.

Note: When moving the files to the “fail” location Camel will handle the error and will not pick up the file again.

include

null

Is used to include files, if filename matches the regex pattern (matching is case in-sensitive from Camel 2.17 onward).

exclude

null

Is used to exclude files, if filename matches the regex pattern (matching is case in-sensitive from Camel 2.17 onward).

antInclude

null

Camel 2.10: Ant style filter inclusion, for example antInclude=*/.txt. Multiple inclusions may be specified in comma-delimited format. See below for more details about ant path filters.

antExclude

null

Camel 2.10: Ant style filter exclusion. If both antInclude and antExclude are used, antExclude takes precedence over antInclude. Multiple exclusions may be specified in comma-delimited format. See below for more details about ant path filters.

antFilterCaseSensitive

true

Camel 2.11: Ant style filter which is case sensitive or not.

idempotent

false

Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.

idempotentKey

Expression

Camel 2.11: To use a custom idempotent key. By default the absolute path of the file is used. You can use the File Language, for example to use the file name and file size, you can do:

idempotentKey=${file:name}-${file:size}

idempotentRepository

null

A pluggable repository org.apache.camel.spi.IdempotentRepository which by default use MemoryMessageIdRepository if none is specified and idempotent is true.

inProgressRepository

memory

A pluggable in-progress repository org.apache.camel.spi.IdempotentRepository . The in-progress repository is used to account the current in progress files being consumed. By default a memory based repository is used.

filter

null

Pluggable filter as a org.apache.camel.component.file.GenericFileFilter class. Will skip files if filter returns false in its accept() method. More details in section below.

filterDirectory

null

Camel 2.18: Filters the directory based on Simple language. For example to filter on current date, you can use a simple date pattern such as ${date:now:yyyMMdd}.

filterFile

null

Camel 2.18: Filters the file based on Simple language. For example to filter on file size, you can use ${file}:size > 5000.

shuffle

false

Camel 2.16: To shuffle the list of files (sort in random order).

sorter

null

Pluggable sorter as a java.util.Comparator<org.apache.camel.component.file.GenericFile> class.

sortBy

null

Built-in sort using the File Language. Supports nested sorts, so you can have a sort by file name and as a 2nd group sort by modified date. See sorting section below for details.

readLock

none

Used by consumer, to only poll the files if it has exclusive read-lock on the file e.g., the file is not in-progress or being written. Camel will wait until the file lock is granted.

This option provides the built-in strategies:

• none is for no read locks at all.
• markerFile Camel creates a marker file fileName.camelLock and then holds a lock on it. This option is not available for the FTP component.
• changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec. to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency. This option is only avail for the FTP component from Camel 2.8 onward. Note: from Camel 2.10.1 onward the FTP option fastExistsCheck can be enabled to speedup this readLock strategy, if the FTP server support the LIST operation with a full file name (some servers may not).
• fileLock is for using java.nio.channels.FileLock. This option is not avail for the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks.
• rename is for using a try to rename the file as a test if we can get exclusive read-lock.
• idempotent Camel 2.16 (only file component) is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.

Warning: most of the read lock strategies are not suitable for use in clustered mode. That is, you cannot have multiple consumers attempting to read the same file in the same directory. In this case, the read locks will not function reliably. The idempotent read lock supports clustered reliably if you use a cluster aware idempotent repository implementation such as from Hazelcast Component or Infinispan.

readLockTimeout

10000

Optional timeout in milliseconds for the readLock, if supported. If the read-lock could not be granted and the timeout triggered, then Camel will skip the file. At next poll Camel, will try the file again, and this time maybe the read-lock could be granted. Use a value of 0 or lower to indicate forever. In Camel 2.0 the default value is 0. Starting with Camel 2.1 the default value is 10000. Currently fileLock, changed and rename support the timeout.

Note: for FTP the default readLockTimeout value is 20000 instead of 10000. The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that ample time is allowed for the read lock process to try to grab the lock before the timeout was hit.

readLockCheckInterval

1000

Camel 2.6: Interval in milliseconds for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the changed read lock, you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. For FTP the default readLockCheckInterval is 5000. The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that ample time is allowed for the read lock process to try to grab the lock before the timeout was hit.

readLockMinLength

1

Camel 2.10.1: This option applied only for readLock=changed. This option allows you to configure a minimum file length. By default Camel expects the file to contain data, and thus the default value is 1. You can set this option to zero, to allow consuming zero-length files.

readLockMinAge

0

Camel 2.15: This option applies only to readLock=change. This option allows you to specify a minimum age a file must be before attempting to acquire the read lock. For example, use readLockMinAge=300s to require that the file is at least 5 minutes old. This can speedup the poll when the file is old enough as it will acquire the read lock immediately. Notice for FTP: file timestamps reported by FTP servers are often reported with resolution of minutes, so readLockMinAge parameter should be defined in minutes, e.g. 60000 for 1 minute. Notice that Camel supports specifying this as 60s, or 1m, etc.

readLockLoggingLevel

WARN

Camel 2.12: Logging level used when a read lock could not be acquired. By default a WARN is logged. You can change this level, for example to OFF to not have any logging.

This option is only applicable for the readLock types:

• changed
• fileLock
• rename

readLockMarkerFile

true

Camel 2.14: Whether to use marker file with the changed, rename, or exclusive read lock types. By default a marker file is used as well to guard against other processes picking up the same files. This behavior can be turned off by setting this option to false. For example if you do not want to write marker files to the file systems by the Camel application.

readLockRemoveOnRollback

true

Camel 2.16: This option applied only for readLock=idempotent. This option allows to specify whether to remove the file name entry from the idempotent repository when processing the file failed and a rollback happens. If this option is false, then the file name entry is confirmed (as if the file did a commit).

readLockRemoveOnCommit

false

Camel 2.16: This option applied only for readLock=idempotent. This option allows to specify whether to remove the file name entry from the idempotent repository when processing the file succeeded and a commit happens. By default the file is not removed which ensures that any race-condition do not occur so another active node may attempt to grab the file. Instead the idempotent repository may support eviction strategies that you can configure to evict the file name entry after X minutes - this ensures no problems with race conditions.

readLockDeleteOrphanLockFiles

true

Camel 2.16: Whether or not read lock with marker files should upon startup delete any orphan read lock files, which may have been left on the file system, if Camel was not properly shutdown (such as a JVM crash). If turning this option to false then any orphaned lock file will cause Camel to not attempt to pickup that file, this could also be due another node is concurrently reading files from the same shared directory.

directoryMustExist

false

Camel 2.5: Similar to startingDirectoryMustExist but this applies during polling recursive sub directories.

doneFileName

null

Camel 2.6: If provided, Camel will only consume files if a done file exists. This option configures what file name to use. Either you can specify a fixed name. Or you can use dynamic placeholders. The done file is always expected in the same folder as the original file. See using done file and writing done file sections for examples.

exclusiveReadLockStrategy

null

Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation.

maxMessagesPerPoll

0

An integer to define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disable it. See more details at Batch Consumer.

Notice: If this option is in use then the File and FTP components will limit before any sorting. For example if you have 100000 files and use maxMessagesPerPoll=500, then only the first 500 files will be picked up, and then sorted. You can use the eagerMaxMessagesPerPoll option and set this to false to allow to scan all files first and then sort afterwards.

eagerMaxMessagesPerPoll

true

Camel 2.9.3: Allows for controlling whether the limit from maxMessagesPerPoll is eager or not. If eager then the limit is during the scanning of files. Where as false would scan all files, and then perform sorting. Setting this option to false allows for sorting all files first, and then limit the poll. Mind that this requires a higher memory usage as all file details are in memory to perform the sorting.

minDepth

0

Camel 2.8: The minimum depth to start processing when recursively processing a directory. Using minDepth=1 means the base directory. Using minDepth=2 means the first sub directory.

This option is supported by FTP consumer from Camel 2.8.2, 2.9 onward.

maxDepth

Integer.MAX_VALUE

Camel 2.8: The maximum depth to traverse when recursively processing a directory. This option is supported by FTP consumer from Camel 2.8.2, 2.9 onward.

processStrategy

null

A pluggable org.apache.camel.component.file.GenericFileProcessStrategy allowing you to implement your own readLock option or similar. Can also be used when special conditions must be met before a file can be consumed, such as a special ready file exists. If this option is set then the readLock option does not apply.

startingDirectoryMustExist

false

Camel 2.5: Whether the starting directory must exist. Mind that the autoCreate option is default enabled, which means the starting directory is normally auto created if it doesn't exist. You can disable autoCreate and enable this to ensure the starting directory must exist. Will thrown an exception if the directory doesn't exist.

pollStrategy

null

A pluggable org.apache.camel.spi.PollingConsumerPollStrategy allowing you to provide your custom implementation to control error handling that may occur during the poll operation but before an Exchange has been created and routed by Camel. In other words the error occurred while the polling was gathering information e.g., access to a file network failed so Camel cannot access it to scan for files.

The default implementation will log the caused exception at WARN level and ignore it.

sendEmptyMessageWhenIdle

false

Camel 2.9: If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

consumer.bridgeErrorHandler

false

Camel 2.10: Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while trying to pickup files, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that by default will be logged at WARN/ERROR level and ignored. See the following section for more details: How to use the Camel error handler to deal with exceptions triggered outside the routing engine.

scheduledExecutorService

null

Camel 2.10: Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. This option allows you to share a thread pool among multiple file consumers.

scheduler

null

Camel 2.12: To use a custom scheduler to trigger the consumer to run. See more details at Polling Consumer, for example there is a Quartz2, and Spring based scheduler that supports CRON expressions.

backoffMultiplier

0

Camel 2.12: To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

For more details see: Polling Consumer.

backoffIdleThreshold

0

Camel 2.12: The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

backoffErrorThreshold

0

Camel 2.12: The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

onCompletionExceptionHandler

Camel 2.16: To use a custom org.apache.camel.spi.ExceptionHandler to handle any thrown exceptions that happens during the file on completion process where the consumer does either a commit or rollback. The default implementation will log any exception at WARN level and ignore.

probeContentType

false

Camel 2.17: Whether to enable probing of the content type. If enable then the consumer uses Files#probeContentType(java.nio.file.Path) to determine the content-type of the file, and store that as a header with key Exchange#FILE_CONTENT_TYPE on the Message.

Camel 2.15-2.16.x the default is true.

extendedAttributes

null

Camel 2.17: To enable gathering extended file attributes through java.nio.file.attribute classes using Files.getAttribute(ava.nio.file.Path, java.lang.String attribute) or Files.readAttributes(ava.nio.file.Path, java.lang.String attributes) depending on the option value. This option supports a comma delimited list of attributes to collect e.g., basic:creationTime, posix:group or simple wildcard e.g., posix:*. If the attribute name is not prefixed, the basic attributes are queried. The result is stored as a header with key CamelFileExtendedAttributes and it is of type Map<String, Object> where the key is the name of the attribute e.g., posix:group and the value is the attributed returned by the call to Files.getAttribute() or Files.readAttributes.