You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 20 Next »

FTP/SFTP Component - Camel 2.0 onwards

This component provides access to remote file systems over the FTP and SFTP protocols.

Using Camel 1.x

If you are using Camel 1.x then see this link for documentation.
This page is only for Camel 2.0 or newer.

URI format

ftp://[username@]hostname[:port]/directoryname[?options]
sftp://[username@]hostname[:port]/directoryname[?options]

Where directoryname represents the underlying directory. Can contain nested folders.
The username is currently only possible to provide in the hostname parameter.

If no username is provided, then anonymous login is attempted using no password.
If no port number is provided, Camel will provide default values according to the protocol (ftp = 21, sftp = 22).

You can append query options to the URI in the following format, ?option=value&option=value&...

URI Options

The options below are exclusive for the FTP2 component.

Unknown macro: {div}

Name

Default Value

Description

password

null

Specifies the password to use to log in to the remote file system.

binary

false

Specifies the file transfer mode, BINARY or ASCII. Default is ASCII (false).

localWorkDirectory

null

When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial, if you consume a very big remote file and thus can conserve memory. See below for more details.

passiveMode

false

FTP only: Specifies whether to use passive mode connections. Default is active mode {false).

ftpClientConfig

null

FTP only: Reference to a bean in the registry as a org.apache.commons.net.ftp.FTPClientConfig class. Use this option if you need to configure the client according to the FTP Server date format, locale, timezone, platform etc. See the javadoc FTPClientConfig for more documentation.

knownHostsFile

null

SFTP only: Sets the known_hosts file, so that the SFTP endpoint can do host key verification.

privateKeyFile

null

SFTP only: Set the private key file to that the SFTP endpoint can do private key verification.

privateKeyFilePassphrase

null

SFTP only: Set the private key file passphrase to that the SFTP endpoint can do private key verification.

maximumReconnectAttempts

3

Specifies the maximum reconnect attempts Camel performs when it tries to connect to the remote FTP server. Use 0 to disable this behavior.

reconnectDelay

1000

Delay in millis Camel will wait before performing a reconnect attempt.

ftpClient

null

Camel 2.1: Allows you to use a custom org.apache.commons.net.ftp.FTPClient instance.

ftpClientConfig

null

Camel 2.1: Allows you to use a custom org.apache.commons.net.ftp.FTPClientConfig instance.

You can configure additional options on the ftpClient and ftpClientConfig from the URI directly by using the ftpClient. or ftpClientConfig. prefix.

For example to set the setDataTimeout on the FTPClient to 30 seconds you can do:

from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000").to("bean:foo");

You can mix and match and have use both prefixes, for example to configure date format or timezones.

from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000&ftpClientConfig.serverLanguageCode=fr").to("bean:foo");

You can have as many of these options as you like.

See the documentation of the Apache Commons FTP FTPClientConfig for possible options and more details.
And as well for Apache Commons FTP FTPClient.

More URI options

See File2 as all the options there also applies for this component.

Examples

ftp://someone@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true
ftp://someoneelse@someotherftpserver.co.uk:12049/reports/2008/password=secret&binary=false
ftp://publicftpserver.com/download

FTP Consumer does not support concurrency

The FTP consumer (with the same endpoint) does not support concurrency (the backing FTP client is not thread safe).
You can use multiple FTP consumers to poll from different endpoints. It is only a single endpoint that does not support concurrent consumers.

The FTP producer does not have this issue, it supports concurrency.

In the future we will add consumer pooling to Camel to allow this consumer to support concurrency as well.

More information

This component is an extension of the File2 component. So there are more samples and details on the File2 component page.

limitations

The option readLock can be used to force Camel not to consume files that is currently in the progress of being written. However, this option is turned off by default, as it requires that the user has write access. There are other solutions to avoid consuming files that are currently being written over FTP; for instance, you can write to a temporary destination and move the file after it has been written.

The ftp producer does not support appending to existing files. Any existing files on the remote server will be deleted before the file is written.

Message Headers

The following message headers can be used to affect the behavior of the component

Header

Description

CamelFileName

Specifies the output file name (relative to the endpoint directory) to be used for the output message when sending to the endpoint. If this is not present and no expression either, then a generated message ID is used as the filename instead.

CamelFileNameProduced

The actual absolute filepath (path + name) for the output file that was written. This header is set by Camel and its purpose is providing end-users the name of the file that was written.

CamelFileBatchIndex

Current index out of total number of files being consumed in this batch.

CamelFileBatchSize

Total number of files being consumed in this batch.

CamelFileHost

The remote hostname.

CamelFileLocalWorkPath

Path to the local work file, if local work directory is used.

Using Local Work Directory

Camel supports consuming from remote FTP servers and downloading the files directly into a local work directory. This avoids reading the entire remote file content into memory as it is streamed directly into the local file using FileOutputStream.

Camel will store to a local file with the same name as the remote file, though with .inprogress as extension while the file is being downloaded. Afterwards, the file is renamed to remove the .inprogress suffix. And finally, when the Exchange is complete the local file is deleted.

So if you want to download files from a remote FTP server and store it as files then you need to route to a file endpoint such as:

from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");

Optimization by renaming work file

The route above is ultra efficient as it avoids reading the entire file content into memory. It will download the remote file directly to a local file stream. The java.io.File handle is then used as the Exchange body. The file producer leverages this fact and can work directly on the work file java.io.File handle and perform a java.io.File.rename to the target filename. As Camel knows it's a local work file, it can optimize and use a rename instead of a file copy, as the work file is meant to be deleted anyway.

Samples

In the sample below we set up Camel to download all the reports from the FTP server once every hour (60 min) as BINARY content and store it as files on the local file system.

Error formatting macro: snippet: java.lang.NullPointerException

And the route using Spring DSL:

  <route>
     <from uri="ftp://scott@localhost/public/reports?password=tiger&amp;binary=true&amp;delay=60000"/>
     <to uri="file://target/test-reports"/>
  </route>

Consuming a remote FTP server triggered by a route

The FTP consumer is built as a scheduled consumer to be used in the from route. However, if you want to start consuming from an FTP server triggered within a route.

Notice we use the Content Enricher EIP with the pollEnrich DSL to consume the FTP file:

from("seda:start")
   // set the filename in FILE_NAME header so Camel know the name of the remote file to poll
   .setHeader(Exchange.FILE_NAME, header("myfile"))
   .pollEnrich("ftp://admin@localhost:21/getme?password=admin&binary=false")
   .to("mock:result");

Filter using org.apache.camel.component.file.GenericFileFilter

Camel supports pluggable filtering strategies. This strategy it to use the build in org.apache.camel.component.file.GenericFileFilter in Java. You can then configure the endpoint with such a filter to skip certain filters before being processed.

In the sample we have build our own filter that only accepts files starting with report in the filename.

Error formatting macro: snippet: java.lang.NullPointerException

And then we can configure our route using the filter attribute to reference our filter (using # notation) that we have defines in the spring XML file:

   <!-- define our sorter as a plain spring bean -->
   <bean id="myFilter" class="com.mycompany.MyFileFilter"/>

  <route>
    <from uri="ftp://someuser@someftpserver.com?password=secret&amp;filter=#myFilter"/>
    <to uri="bean:processInbox"/>
  </route>

Filtering using ANT path matcher

The ANT path matcher is a filter that is shipped out-of-the-box in the camel-spring jar. So you need to depend on camel-spring if you are using Maven.
The reason is that we leverage Spring's AntPathMatcher to do the actual matching.

The file paths are matched with the following rules:

  • ? matches one character
  • * matches zero or more characters
  • ** matches zero or more directories in a path

The sample below demonstrates how to use it:

Error formatting macro: snippet: java.lang.NullPointerException

Debug logging

This component has log level TRACE that can be helpful if you have problems.

See Also

  • No labels