THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
| Wiki Markup |
|---|
h2. FTP/SFTP Component
This component provides access to remote file systems over the FTP and SFTP protocols.
h3. URI format
{code}
ftp://[username@]hostname[:port]/directoryname[?options]
sftp://[username@]hostname[:port]/directoryname[?options]
{code}
Where *directoryname* represents the underlying directory. Can contain nested folders.
The *username* is currently only possible to provide in the hostname parameter.
If no *username* is provided, then {{anonymous}} login is attempted using no password.
If no *port* number is provided., Camel will provide default values according to the protocol. (ftp = 21, sftp = 22)
h4. Examples
{{ftp://someone@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true}}
{{ftp://someoneelse@someotherftpserver.co.uk:12049/reports/2008/password=secret&binary=false}}
{{ftp://publicftpserver.com/download}}
{warning:title=FTP Consumer does not support concurrency}
The FTP consumer (with the same endpoint) does not support concurrency (the backing FTP client is not thread safe).
You can use multiple FTP consumers to poll from different endpoints. Its only a single endpoint that does not support concurrent consumers.
The FTP producer does *not* have this issue, it supports concurrency.
In the future we will [add consumer pooling to Camel|https://issues.apache.org/activemq/browse/CAMEL-1682] to allow this consumer to support concurrency as well.
{warning}
{tip:title=More information}
This component is an extension of the [File2] component. So there are more samples and details on the [File2] component page.
{tip}
h3. URI Options
The options below are exclusive for the [FTP2] component.
{div:class=confluenceTableSmall}
|| Name || Default Value || Description ||
| password | null | specifies the password to use to login to the remote file system |
| binary | false | specifies the file transfer mode BINARY or ASCII. Default is ASCII. |
| localWorkDirectory | null | When consuming a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This benefits if you consume very big remote file and thus can preserve memory usage. See below for more details. |
| passiveMode | false | *FTP only*: Set whether to use passive mode connections. Default is active. |
| ftpClientConfig | null | *FTP only*: Reference to a bean in the registry as a {{[org.apache.commons.net.ftp.FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} class. Use this option if you need to configure the client according to the FTP Server date format, locale, timezone, platform etc. See the javadoc {{[FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} for more documentation.
| knownHostsFile | null | *SFTP only:* Sets the known_hosts file so that the SFTP endpoint can do host key verification. |
| privateKeyFile | null | *SFTP only:* Set the private key file to that the SFTP endpoint can do private key verification. |
| privateKeyFilePassphrase | null | *SFTP only:* Set the private key file passphrase to that the SFTP endpoint can do private key verification. |
{div}
h3. More URI options
{info:title:More options}
See [File2] as all the options there also applies for this component.
{info.
You can append query options to the URI in the following format, {{?option=value&option=value&...}}
h3. URI Options
The options below are exclusive for the [FTP2] component.
{div:class=confluenceTableSmall}
|| Name || Default Value || Description ||
| {{password}} | {{null}} | Specifies the password to use to log in to the remote file system. |
| {{binary}} | {{false}} | Specifies the file transfer mode, BINARY or ASCII. Default is ASCII ({{false}}). |
| {{localWorkDirectory}} | {{null}} | When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial, if you consume a very big remote file and thus can conserve memory. See below for more details. |
| {{passiveMode}} | {{false}} | *FTP only*: Specifies whether to use passive mode connections. Default is active mode {{{false}}). |
| {{ftpClientConfig}} | {{null}} | *FTP only*: Reference to a bean in the registry as a {{[org.apache.commons.net.ftp.FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} class. Use this option if you need to configure the client according to the FTP Server date format, locale, timezone, platform etc. See the javadoc {{[FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} for more documentation.
| {{knownHostsFile}} | {{null}} | *SFTP only:* Sets the {{known_hosts}} file, so that the SFTP endpoint can do host key verification. |
| {{privateKeyFile}} | {{null}} | *SFTP only:* Set the private key file to that the SFTP endpoint can do private key verification. |
| {{privateKeyFilePassphrase}} | {{null}} | *SFTP only:* Set the private key file passphrase to that the SFTP endpoint can do private key verification. |
{div}
h3. More URI options
{info:title:More options}
See [File2] as all the options there also applies for this component.
{info}
h4. Examples
{{ftp://someone@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true}}
{{ftp://someoneelse@someotherftpserver.co.uk:12049/reports/2008/password=secret&binary=false}}
{{ftp://publicftpserver.com/download}}
{warning:title=FTP Consumer does not support concurrency}
The FTP consumer (with the same endpoint) does not support concurrency (the backing FTP client is not thread safe).
You can use multiple FTP consumers to poll from different endpoints. It is only a single endpoint that does not support concurrent consumers.
The FTP producer does *not* have this issue, it supports concurrency.
In the future we will [add consumer pooling to Camel|https://issues.apache.org/activemq/browse/CAMEL-1682] to allow this consumer to support concurrency as well.
{warning}
{tip:title=More information}
This component is an extension of the [File2] component. So there are more samples and details on the [File2] component page.
{tip}
h4. limitations
The option *readLock* can be used to force Camel *not* to consume files that is currently in the progress of being written. However, this option is default turned off by default, as it requires that the user has write access. There are other solutions to avoid consuming files that are currently being written over FTP,; for instance, you can write theto a temporary destination and move the file after it has been written.
The ftp producer does *not* support appending, to existing files. Any existing files on the remote server will be deleted before the file is written.
h3. Message Headers
The following message headers can be used to affect the behavior of the component
|| Header || Description ||
| {{CamelFileName}} | Specifies the output file name (relative to the endpoint directory) to be used for the output message when sending to the endpoint. If this is not present and no expression either, then a generated message IdID is used as the filename instead. |
| {{CamelFileNameProduced}} | The actual absolute filepath (path + name) for the output file that was written. This header is set by Camel and its purpose is providing end-users the name of the file that was written. |
| {{CamelFileBatchIndex}} | Current index out of total number of files being consumed in this batch. |
| {{CamelFileBatchSize}} | Total number of files being consumed in this batch. |
| {{CamelFileHost}} | The remote hostname. |
| {{CamelFileLocalWorkPath}} | Path to the local work file, if local work directory is used. |
h3. Using Local Work Directory
Camel supports consuming from remote FTP servers and downloading the files directly into a local work directory. This avoids reading the entire remote file content into memory as it itsis streamed directly into the local file using {{FileOutputStream}}.
Camel will store the intoto a local file with the same name as the remote file, thoughtthough with {{.progress}} as extension while the file is being downloaded. Afterwards, the file is renamed to remove the {{.inprogress}} prefixsuffix. And finally, when the [Exchange] is complete the local file is deleted.
So if you want to download files from a remote FTP server and store it as files then you need to route to a file endpoint such as:
{code:java}
from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
{code}
{tip:title=Optimization by renaming work file}
The route above is ultra efficient as it avoids reading the entire file content into memory. It will download the remote file directly to a local file stream. The {{java.io.File}} handle is then used as the [Exchange] body. The file producer leverages this fact and can work directly on the work file {{java.io.File}} handle and perform a {{java.io.File.rename}} to the target filename. As Camel knowknows it's a local work file, it can optimize and use a rename instead of a file copy, as the work file is meant to be deleted anyway.
{tip}
h3. Samples
In the sample below we set setupup Camel to download all the reports from the FTP server once every hour (60 min) as BINARY content and store it as files on the local file system.
{snippet:id=e1|lang=java|url=camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FromFtpToBinarySampleTest.java}
And the route using Spring DSL:
{code:xml}
<route>
<from uri="ftp://scott@localhost/public/reports?password=tiger&binary=true&delay=60000"/>
<to uri="file://target/test-reports"/>
</route>
{code}
h4. Consuming a remote FTP server triggered by a route
The FTP consumer is buildbuilt as a scheduled consumer to be used in the *from* route. However, if you want to start consuming from aan FTP server triggered within a route, it's a bit cumbersome to do this in Camel 1.x as opposed to Camel 2.0 where its supported directly in the DSL. However it's possible to do so in Camel 1.x, as this code below demonstrates.
h5. Camel 1.x
In the sample we have a [Seda] queue where a message arrives that holds a message containing a filename to poll from a remote FTP server. So we set setupup a basic FTP urlURL as:
{snippet:id=e1|lang=java|url=camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FromQueueThenConsumeFtpToMockTest.java}
And then we have the route where we use a [Processor] within the route, so we can use Java code. In this Java code, we create the ftp consumer that downloads the file we want. And after the download we can get the content of the file and put it in the original exchange that continues being routed. As this is based on an unit test it routes to a [Mock] endpoint.
{snippet:id=e2|lang=java|url=camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FromQueueThenConsumeFtpToMockTest.java}
h5. Camel 2.0
The same sample would be like this in Camel 2.0 where we can use the [Content Enricher] EIP with the {{pollEnrich}} DSL:
{code}
from("seda:start")
// set the filename in FILE_NAME header so Camel know the name of the remote file to poll
.setHeader(Exchange.FILE_NAME, header("myfile"))
.pollEnrich("ftp://admin@localhost:21/getme?password=admin&binary=false")
.to("mock:result");
{code}
h3. Filter using {{org.apache.camel.component.file.GenericFileFilter}}
Camel supports pluggable filtering strategies. This strategy it to use the build in {{org.apache.camel.component.file.GenericFileFilter}} in Java. You can then configure the endpoint with such a filter to skip certain filters before being processed.
In the sample we have build our own filter that only accepts files starting with report in the filename.
{snippet:id=e1|lang=java|url=camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FromFtpRemoteFileFilterTest.java}
And then we can configure our route using the *filter* attribute to reference our filter (using {{\#}} notation) that we have defines in the spring XML file:
{code:xml}
<!-- define our sorter as a plain spring bean -->
<bean id="myFilter" class="com.mycompany.MyFileFilter"/>
<route>
<from uri="ftp://someuser@someftpserver.com?password=secret&filter=#myFilter"/>
<to uri="bean:processInbox"/>
</route>
{code}
h3. Filtering using ANT path matcher
The ANT path matcher is a filter that is shipped out-of-the-box in the *camel-spring* jar. So you need to depend on *camel-spring* if you are using Maven.
The reasonsreason is that we leverage Spring's [AntPathMatcher|http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/util/AntPathMatcher.html] to do the actual matching.
The file paths isare matched with the following rules:
- {{?}} matches one character
- {{*}} matches zero or more characters
- {{**}} matches zero or more directories in a path
The sample below demonstrates how to use it:
{snippet:id=example|lang=xml|url=camel/trunk/tests/camel-itest/src/test/resources/org/apache/camel/itest/ftp/SpringFileAntPathMatcherRemoteFileFilterTest-context.xml}
h3. Debug logging
This component has log level *TRACE* that can be helpful if you have problems.
{include:Endpoint See Also}
- [File2] |