This documentation has not been fully vetted. If you use it and find errors please fix them, or leave a comment and the authors will apply fixes.
Once this document has been fully vetted this message will be removed. -Thanks!
Download and Install
- Download a Released tarball from the Downloads page.
- Untar it
- cd into the apache-oodt-{version}-src folder
- mvn package
Now the required Maven artifacts have been downloaded and installed into your local maven m2 repo. Time for deployment to your local machine or to another server.
Deployment
- cd into pushpull/target
- Copy the tarball (cas-pushpull-{version}-dist.tar.gz) to your deployment location
- untar the tarball and you will have folder named cas-pushpull-{version} with the following directory structure
/bin /etc /lib /logs /policy
Configuration
Basic Configuration
This is a set of configuration that must be completed to get the Push/Pull framework setup. These setups are required for even the most basic installations. We will cover deployment specific setup/configuration in the next section.
This documentation has been written assuming the environment variable CAS_PP_HOME has been set to the directory where you have untar'd the pushpull component. Several configuration properties require a full file path. Just be sure to replace the CAS_PP_HOME with a value that is applicable to your deployment, or export that environment variable and use the following config.
The following Sub-Sections will reference the path to each file that needs to be edited, and each file will be followed by a block showing what changes need to be made
[CAS_PP_HOME]/etc/push_pull_framework.properties
line 21 #external configuration files 22 org.apache.oodt.cas.pushpull.config.external.properties.files=[CAS_PP_HOME]/etc/default.properties 35 # ingester filemgr url 36 org.apache.oodt.cas.filemgr.url= 61 #protocolfactory specification for protocol types 62 org.apache.oodt.cas.pushpull.config.protocolfactory.info.files=[CAS_PP_HOME]/policy/ProtocolFactoryInfo.xml 69 #parser to retrievalmethod map 70 org.apache.oodt.cas.pushpull.config.parser.info.files=[CAS_PP_HOME]/policy/ParserToRetrievalMethodMap.xml 71 72 #unique metadata element info 73 org.apache.oodt.cas.pushpull.config.type.detection.file=[CAS_PP_HOME]/policy/mimetypes.xml 74 75 #directory below which all data file will be downloaded to 76 org.apache.oodt.cas.pushpull.data.files.base.staging.area=[CAS_PP_HOME]/staging
Specific Configuration(s)
Due to the limitless combinations of protocols and remote data archives the following list of example is NOT exhaustive and is intended to give you working examples. Each configuration will begin with a summary description of the problem being solved, then it will be followed with config/setups needed to solve the problem.
Example of Connecting to a Remote FTP Server to Retrieve All *.he5 Files
Connection Protocol: FTP
Root Path: ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/
Password Required: NO
Download (All or Subset)?: All
Examples of full path:
ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/2004.09.20/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5 ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/2005.05.21/TES-Aura_L2-CO2-Nadir_r0000002931_F06_08.he5
[CAS_PP_HOME]/policy/mimetypes.xml
[CAS_PP_HOME]/policy/ExternalSources.xml
Purpose: This file contains a list of External Data Sources such as FTP Servers. The login.alias attribute will be used within the RemoteSpecs.xml
<sources> <source host="e4ftl01.cr.usgs.gov"> <login type="ftp" alias="MOD09GAFtp"> <username>anonymous</username> <password>user@host.com</password> </login> </source> </sources>