...
Within the mimetypes.xml file we need to map a filename pattern (regex or not) to a custom mimetype. Below we have 3 mimetypes, the first 2 are default in pushpull the 3rd is a custom one based on the filenaming of our desired HDF-5 remote files.
Code Block |
---|
|
<mime-info>
<mime-type type="metadata/cas_pushpull">
<glob pattern="*.info.tmp"/>
</mime-type>
<mime-type type="metadata/cas_metadata">
<glob pattern="*.cas"/>
<glob pattern="*.met"/>
</mime-type>
<mime-type type="product/TESLevel2CO2">
<_comment>Level 2 - CO2 Retrivals from TES</_comment>
<glob pattern="TES-Aura_L2-CO2-Nadir_r\d{10}\w{2}\d{2}\w\d{2}\.he5" isregex="true"/>
</mime-type>
</mime-info>
|
...
Code Block |
---|
|
<remoteSpecs>
<aliasSpecs>
<aliasSpec file="[CAS_PP_HOME]/etc/examples/ExternalSources/ExternalSources.xml"/>
</aliasSpecs>
<daemons>
<daemon alias="TESL2CO2" active="yes">
<runInfo firstRunDateTime="2011-12-01T00:00:00Z" period="3m" runOnReboot="yes"/>
<propInfo dir="[CAS_PP_HOME]/etc/examples/DirStructXmlParserFiles">
<propFiles regExp="TESL2CO2\.xml" parser="org.apache.oodt.cas.pushpull.filerestrictions.parsers.DirStructXmlParser"/>
</propInfo>
<dataInfo stagingArea="TESL2CO2" deleteFromServer="no"/>
</daemon>
</daemons>
</remoteSpecs>
|
Wiki Markup |
---|
\[CAS_PP_HOME\]/etc/examples/DirStructXmlParserFiles/TESL2CO2.xml |
Purpose: This file tells pushpull how to parse the remote directory structure. In this example the starting_path is static for all of our remote file paths, but then we have dynamic folders that correspond to a YYYY.MM.DD format so we have a simple regex to pushpull will dig down into each subfolder and will pull out the filename we have declared with another regex.
Within the examples/DirStructXmlParserFiles there are several different examples to learn from.
Code Block |
---|
|
<root>
<dirstruct starting_path="/TES/TL2CO2N.005">
<nofiles/>
<dir name="\d{4}\.\d{2}\.\d{2}"> <!-- regex matching '2004.09.20' -->
<nodirs/>
<!-- regex matching TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5 -->
<file name="TES-Aura_L2-CO2-Nadir_r\d{10}\w{2}\d{2}\w\d{2}\.he5"/>
</dir>
</dirstruct>
</root>
|