...
This example code uses a regular expression to match files on the remote system, so that not all of the files in the source directory are transferred. The filePatternString
is set to ".*/smoke\\d{8}_wkt
.txt". This has the regular expression components of:
- ".*/" matches any path that precedes the filename. The "." is wildcard character, "*" specifies that 0 or more of these may be present. The "/" matches the directory separator.
- "
d{8}" matches 8 digits (4 for the year, 2 for the month number, and 2 for the day of the month). The doubled backslash is to escape a single backslash for Java string literals. "\d" indicates any digit, and "{8}" specifies exactly 8 of the preceding character or pattern. - "
." is an escaped "\." which means a literal period instead of a wild card interpretation of "."
Wiki Markup |
---|
It is important to call the \[http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/impl/DefaultFileSystemManager.html#close() close()\] method of the DefaultFileSystemManager to clean up any temporary files and close all providers. Otherwise the program will appear to hang after downloading files. |
Source Code
The code provided below is for a Maven 2 project.
pom.xml Project File
This example uses Maven2. There is a The pom.xml
to define file defines how the project is built for maven 2:
No Format |
---|
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>gov.noaa.eds.byExample</groupId>
<artifactId>trySimpleVfsSftp</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>trySimpleVfsSftp</name>
<url>http://maven.apache.org</url>
<build>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh-external</artifactId>
<version>1.0-beta-2</version>
</extension>
</extensions>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>gov.noaa.eds.byExample.trySimpleVfsSftp.App</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<configuration>
<tasks>
<java classname="gov.noaa.eds.byExample.trySimpleVfsSftp.App" classpathref="maven.runtime.classpath">
</java>
</tasks>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<!-- Supports VFS SFTP -->
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.23</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>commons-vfs</groupId>
<artifactId>commons-vfs</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
|
Code Listing
Content of src/main/java/gov/noaa/eds/byExample/trySimpleVfsSftp/App.java
...
'''. Be sure to customize the variables near the top of the listing to match your environment if you intend to run this code. The variable to modifiy are host, user, password, remoteDir, filePatternString
and localDir
.
No Format |
---|
/* * App.java */ package gov.noaa.eds.byExample.trySimpleVfsSftp; import java.io.File; import java.util.regex.Pattern; import org.apache.commons.vfs.AllFileSelector; import org.apache.commons.vfs.FileObject; import org.apache.commons.vfs.FileSystemException; import org.apache.commons.vfs.FileSystemManager; import org.apache.commons.vfs.FileSystemOptions; import org.apache.commons.vfs.FileType; import org.apache.commons.vfs.UserAuthenticator; import org.apache.commons.vfs.VFS; import org.apache.commons.vfs.auth.StaticUserAuthenticator; import org.apache.commons.vfs.impl.DefaultFileSystemConfigBuilder; import org.apache.commons.vfs.impl.DefaultFileSystemManager; import org.apache.commons.vfs.impl.StandardFileSystemManager; import org.apache.commons.vfs.provider.local.LocalFile; /** * Example use of VFS sftp * */ public class App { // Set these variables for your testing environment: private String host = "sftpremote.example.com"; // Remote SFTP hostname private String user = "smokey"; // Remote system login name private String password = "bear"; // Remote system password private String remoteDir = "/data/source/fires/smoke"; // Look for a file path like "smoke20070128_wkt.txt" private String filePatternString = ".*/smoke\\d{8}_wkt\\.txt"; // Local directory to receive file private String localDir = "/extra/data/fires/smoke"; private File localDirFile; private Pattern filePattern; private FileSystemManager fsManager = null; private FileSystemOptions opts = null; private FileObject sftpFile; public static void main(String[] args) { System.out.println("SFTP download"); App app = new App(); app.initialize(); app.process(); app.release(); } // main( String[] args ) /** * Creates the download directory localDir if it * does not exist and makes a connection to the remote SFTP server. * */ public void initialize() { if (localDirFile == null) { localDirFile = new File(localDir); } if (!this.localDirFile.exists()) { localDirFile.mkdirs(); } try { this.fsManager = VFS.getManager(); } catch (FileSystemException ex) { throw new RuntimeException("failed to get fsManager from VFS", ex); } UserAuthenticator auth = new StaticUserAuthenticator(null, this.user, this.password); this.opts = new FileSystemOptions(); try { DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(opts, auth); } catch (FileSystemException ex) { throw new RuntimeException("setUserAuthenticator failed", ex); } this.filePattern = Pattern.compile(filePatternString); } // initialize() /** * Retrieves files that match the specified FileSpec from the SFTP server * and stores them in the local directory. */ public void process() { String startPath = "sftp://" + this.host + this.remoteDir; FileObject[] children; // Set starting path on remote SFTP server. try { this.sftpFile = this.fsManager.resolveFile(startPath, opts); System.out.println("SFTP connection successfully established to " + startPath); } catch (FileSystemException ex) { throw new RuntimeException("SFTP error parsing path " + this.remoteDir, ex); } // Get a directory listing try { children = this.sftpFile.getChildren(); } catch (FileSystemException ex) { throw new RuntimeException("Error collecting directory listing of " + startPath, ex); } search: for (FileObject f : children) { try { String relativePath = File.separatorChar + f.getName().getBaseName(); if (f.getType() == FileType.FILE) { System.out.println("Examining remote file " + f.getName()); if (!this.filePattern.matcher(f.getName().getPath()).matches()) { System.out.println(" Filename does not match, skipping file ." + relativePath); continue search; } String localUrl = "file://" + this.localDir + relativePath; String standardPath = this.localDir + relativePath; System.out.println(" Standard local path is " + standardPath); LocalFile localFile = (LocalFile) this.fsManager.resolveFile(localUrl); System.out.println(" Resolved local file name: " + localFile.getName()); if (!localFile.getParent().exists()) { localFile.getParent().createFolder(); } System.out.println(" ### Retrieving file ###"); localFile.copyFrom(f, new AllFileSelector()); } else { System.out.println("Ignoring non-file " + f.getName()); } } catch (FileSystemException ex) { throw new RuntimeException("Error getting file type for " + f.getName(), ex); } } // for (FileObject f : children) } // process(Object obj) public void release() { ((DefaultFileSystemManager) this.fsManager).close(); } // release() } // class App |
Compiling
Compile the source code with
...
No Format |
---|
java -jar target/trySimpleVfsSftp-1.0-SNAPSHOT-jar-with-dependencies.jar |
Sample Output
No Format |
---|
}}} There should now be files matching the {{{filePatternStringSFTP download Mar 25, 2008 1:00:44 PM org.apache.commons.vfs.VfsLog info INFO: Using "/tmp/vfs_cache" as temporary files store. SFTP connection successfully established to sftp://sftpremote.example.com/data/source/fires/smoke Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/README.txt Filename does not match, skipping file ./README.txt Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/smoke20070328_wkt.txt Standard local path is /extra/data/fires/smoke/smoke20070328_wkt.txt Resolved local file name: file:///extra/data/fires/smoke/smoke20070328_wkt.txt ### Retrieving file ### Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/smoke20070329.txt Filename does not match, skipping file ./smoke20070329.txt |
There should now be files matching the filePatternString
in the local machine directory "/extra/data/fires/smoke".