Warning | ||||||||
---|---|---|---|---|---|---|---|---|
Pushpull is deprecated... see
|
Table of Contents |
---|
Introduction
OODT's PushPull component framework provides a client architecture for accessing an array of remote resources. This component is used to pull from remote resources and push to local ones. It is typically used in conjunction with the CAS Crawler component. An example use case would be pulling data products from a remote FTP service and pushing them to a local staging area from which the CAS Crawler then then inject them into the File Manager.
Download and Install
- Download a Released tarball/zip from the Downloads page. (http://oodt.apache.org/downloads/download)
- Uncompress it
- cd into the apache-oodt-{version} folder
- mvn package
...
Connection Protocol: FTP
Root Path: ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/
Password Required: NO
Download (All or Subset)?: All
Examples of full path to where the data resides on the FTP server:
Code Block |
---|
ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/2004.09.20/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5 ftp://l4ftl01.larc.nasa.gov/TES/TL2CO2N.005/2005.05.21/TES-Aura_L2-CO2-Nadir_r0000002931_F06_08.he5 |
...
- cd $CAS_PP_HOME/bin
- The two options listed below:
- Export 2 env vars
Replace the CAS_PP_RESOURCES and DAEMONLAUNCHER_PORT with static values
Code Block borderStyle dashed title [CAS_PP_HOME]/bin/pushpull borderStyledashed line 25 ${JAVA_HOME}/bin/java \ 26 -cp ${LIB_DEPS} -Dcom.sun.management.jmxremote \ 27 -Djava.util.logging.config.file=../etc/logging.properties \ 28 -Djavax.net.ssl.trustStore=${CAS_PP_RESOURCES}/jssecacerts \ 29 org.apache.oodt.cas.pushpull.daemon.DaemonLauncher \ 30 --rmiRegistryPort ${DAEMONLAUNCHER_PORT} \ 31 --propertiesFile ${CAS_PP_RESOURCES}/push_pull_framework.properties \ 32 --remoteSpecsFile ${CAS_PP_RESOURCES}/examples/RemoteSpecsFiles/RemoteSpecs.xml # You can leave this file unchanged by merely exporting the following env vars (bash shell) export CAS_PP_RESOURCES=$CAS_PP_HOME/etc export DAEMONLAUNCHER_PORT=9012 # Or you can always use this config and not setup env vars line 25 ${JAVA_HOME}/bin/java \ 26 -cp ${LIB_DEPS} -Dcom.sun.management.jmxremote \ 27 -Djava.util.logging.config.file=${CAS_PP_HOME}/etc/logging.properties \ 28 -Djavax.net.ssl.trustStore=${CAS_PP_HOME}/etc/jssecacerts \ 29 org.apache.oodt.cas.pushpull.daemon.DaemonLauncher \ 30 --rmiRegistryPort 9012 \ 31 --propertiesFile ${CAS_PP_HOME}/etc/push_pull_framework.properties \ 32 --remoteSpecsFile ${CAS_PP_HOME}/etc/examples/RemoteSpecsFiles/RemoteSpecs.xml
- ./pushpull
...
1. You will need to have a fileManager that pushpull can inspect to see if the product has been ingested into the archive.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# ingester filemgr url org.apache.oodt.cas.filemgr.url=http://localhost:9000 |
2. Then you just configure the RemoteSpecs.xml file and update the <dataInfo> element and set queryElement="Filename" within the <daemon> block. If you have multiple daemon's configured you will have to configure each one.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<dataInfo stagingArea="MOD09GA-NRT" deleteFromServer="no" queryElement="Filename"/> </daemon> |
...