THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Path | Description |
|---|---|
/data/archive | This is the root of where the filemgr will store its archived products |
/data/staging | This directory will be monitored by the crawler. Products to be ingested should be placed here |
/data/catalog | In a configuration that uses Lucene as a back end this directory holds the contents of that index |
/data/work | ... |
/data/failure | Any products that have failed ingestion will be placed here along with any metadata files. |
/bin | Contains system level scripts to start, stop, restart the OODT infrastructure |
/components | The guts of what make the data management system work |
/components/crawler | The crawler deployment for your data management system (i.e. policy, scripts, and configuration). This component is responsible for monitoring the staging area |
/components/filemgr | The filemgr deployment for your data management system (i.e. policy, scripts, and configuration). This component catalogs and archives products into the archive area. |
/components/workflow | The workflow deployment for your data management system (i.e. policy, scripts, and configuration). This component orchestrates any processing that may need to be done on your products |
/components/extensions | This is a sandbox area to test out metadata extractors, versioners, actions, etc. that you have developed to extend the functionality of the existing OODT framework. |
/confetc | System wide configuration |
...
*/$DEPLOYMENT_BASE_DIR/components - This will contain a single folder for each component. Initially this will only contain the 3 components we have selected to start this process, but as more components are added they will be added in here.
/$DEPLOYMENT_BASE_DIR/confetc - This will contain configuration and properties files which apply to several components. This should (like the bin dir) give users a single directory they can go into to configure the associate components.
...
| Code Block |
|---|
/$DEPLOYMENT_BASE_DIR
/crawler
/src/main/resources
/policy
/bin
/etc
/filemgr
/src/main/resources
/policy
/oodt
/bin
/etc
/workflow
/src/main/resources
/policy
/bin
/etc
/webapps
/fmprod
/fmbrowser
/wmonitor
/curator
/extensions
/src/main
/java
/<package>
/extractor
/versioner
/task
/action
/python
/distribution
/src/main/resources
/bin
/oodt-pcs
/etc
/oodt.properties
|
Source Path Details
Path | Description |
|---|---|
/crawler | Project specific crawler configuration, policy, and scripts |
/filemgr | Project specific filemgr configuration, policy, and scripts |
/workflow | Project specific workflow configuration, policy, and scripts |
/webapps | Web Applications from Apache OODT |
/extensions | Extensions to the OODT framework to do metadata extraction, archive layout (aka versioner), workflow tasks, crawler actions |
/distribution | Distribution package project for system level build, configuration, and scripts |
Future Work
Once the above is complete our thoughts are that the next items to be incorporated are as follows:
...