Page History

...

1. I had a file manager listening on http://localhost:9000.
2. I've used an external meta data extractor (written in python) to extract data from HDF5 files.
3. MetExtractorProductCrawler example configuration can be found in the source (allows you to specify how the crawler will run your extractor): https://svn.apache.org/repos/asf/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml

...

MetExtractorProductCrawler, using the TikaCmdLineMetExtractor (an easier approach)

...

NOTE: This extractor is only available in 07-SNAPSHOT+

Without having to create your own custom MetExtractor, you can leverage OODT's Tika extractor do automatically extract as much metadata as it can gather for you. The only thing you need to do is to specify a configuration file, and specify which ProductType you want your products ingested to. Below are examples of the steps you could perform:

Invocation command:
./crawler_launcher
--filemgrUrl http://localhost:9000Image Added
--operation --launchMetCrawler
--clientTransferer org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
--productPath /usr/local/meerkat/data/staging/products/hdf5
--metExtractor org.apache.oodt.cas.metadata.extractors.TikaCmdLineMetExtractor
--metExtractorConfig /usr/local/meerkat/extractors/tikaextractor/tikaextractor.config

Associated configuration file:

ProductType=MyCustomProductType

AutoDetectProductCrawler

To get the auto detect product crawler working I ran:

...