...
A few have ben documented below, however additional implementations also exist at the link above so check them out.
External Metadata Extractor
<?xml version="1.0" encoding="UTF-8"?>
<cas:externextractor xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">>
<exec workingDir="">
> <extractorBinPath envReplace="true">[PWD]/extractor</extractorBinPath> <args>extractorBinPath>
<args>
<arg isDataFile="true"/>
> <arg isPath="true">/usr/local/etc/testExtractor.config</arg>arg>
</args>
</exec>
</cas:externextractor>
<args> <arg>arg1</arg> <arg>arg2</arg> <arg>arg3</arg> </args>
- isDataFile="true" - This attribute passes the full path to the product from which metadata is to be extracted as the argument.
- isPath="true" - This attribute passes the argument encoded as a properly formed path (no char-set replacement, etc).
- envReplace="true" - This attribute replaces any part of the value of the argument that is inside brackets ([and ]) with the environment variable matching the text inside the brackets, if such an enviroment variable exists.
The Filename Token Metadata Extractor
In many cases, products that are to be ingested are named with metadata that should be extracted from the product name and cataloged upon ingest. For this type of situation, we have developed theFilenameTokenMetExtractor. This extractor uses a configuration file that specifies, for each metadata element, the index of the start position in the name for this metadata and its character length.
Below is an example configuration file used by the FilenameTokenMetExtractor. It assumes a product name formatted as follows:
MissionName_Date_StartOrbitNumber_StopOrbitNumber.txt
<input> <group name="SubstringOffsetGroup"> <vector name="MissionName"> <element>1</element> <element>11</element> </vector> <vector name="Date"> <element>13</element> <element>4</element> </vector> <vector name="StartOrbitNumber"> <element>18</element> <element>16</element> </vector> <vector name="StopOrbitNumber"> <element>35</element> <element>15</element> </vector> </group> <group name="CommonMetadata"> <scalar name="DataVersion">1.0</scalar> <scalar name="CollectionName">Test</scalar> <scalar name="DataProvider">OODT</scalar> </group> </input>
In this configuration, the FilenameTokenMetExtractor will produce four metadata elements from the product name:MissionName, Date, StartOrbitNumber, and StopOrbitNumber. The first element of each of these groups is the start index (this assumes 1-indexed strings). The second element is the substring length.
Additionally, this configuration specifies that metadata for all products additionally contain three comment metadata elements that are static: DataVersion, CollectionName, and DataProvider.
Metadata Reader Extractor
The MetReaderExtractor, part of the OODT CAS-Metadata project, assumes that a metadata file with then nameing convention "<Product Name>.met" is present in the same directory as the product. This extractor further assumes that the metadata is in the format specified in this document.
Copy And Rewrite Extractor
The CopyAndRewriteExtractor is a metadata extractor, that, like the MetReaderExtractor, assumes that a metadata file exists for the product from which metadata is to be extracted. This extractor reads in the original metadata file and replaces particular metadata values in that metadata file.
The CopyAndRewriteExtractor takes in a configuration file that is a java properties object with the following properties defined:
- numRewriteFields - The number of fields to rewrite within the original metadata file.
- rewriteFieldN - The name(s) of the fields to rewrite in the original metadata file.
- orig.met.file.path - The original path to the metadata file from which to draw the original metadata fields.
- fieldN.pattern - The string specification that details which fields to replace and to use in building the new field value.
An example of the configuration file is given below:
numRewriteFields=2 rewriteField1=ProductType rewriteField2=FileLocation orig.met.file.path=./src/resources/examples/samplemet.xml ProductType.pattern=NewProductType[ProductType] FileLocation.pattern=/new/loc/[FileLocation]
In ths example configuration, two metadata elements will be rewritten, ProductType and FileLocation. The original metadata file is located on at ./src/resources/examples/samplemet.xml. The Product Type will be rewritten as NewProductType<original ProductType value>. The File location will now be set to/new/location./src/resources/examples/samplemet.xml.