TODO: Examples need to also include changes that are required in the filemanager.properties file to show how file manager must be configured to pull in the policy files.
The CAS File Manager is a great data archive tool that is extremely flexible, but sometimes that flexibility can lead to confusion since you can perform a single task (like defining metadata elements) in various ways. This page's focus is to capture the 'Best Practices' people have found on their projects/experience when creating policy.
To try and keep the confusion to a minimum we will start with a Taxonomy to define some key terms, then jump into some Operational Scenarios. The scenario for all the Ops Scenario's will be built around the idea of cataloging and archiving MEDIA (audio, video, images) since most everyone can relate to these items. Some of the simple examples will focus on a single media format, while the more complex examples will show how to deal with an ever changing media library.
Taxonomy
File Manager Policy - All the *.xml files that the filemanager will use to define metadata
...
Metadata Elements - Data elements that will be cataloged by the File Manager about a product. Metadata Elements must be listed in elements.xml and product-type-element-map.xml.
Operational Scenarios
Simple File Manager Policy
- One set of File Manager Policy
- All of the Data Sets are homogenous
Example: You want to catalog and archive music files. They are logically grouped together by some album, but every song you archive has the same metadata elements. In this case we have a single Data Set called Music, which will be mapped to the default Product Type: GenericFile.
Sample Policy Overview
NOTE: Items that are bold/italic are default policy that come pre-installed with the File Manager and do not need to be edited.
...
- type=GenericFile
- +CAS.ProductId
- +CAS.ProductName
- +CAS.ProductReceivedTime
- +Filename
- +FileLocation
- +ProductType
- +ProductStructure
- +MimeType
- +Album
- +Artist
- +Track_Number
- +Year
- +Title
File Manager Policy with Inheritance
- One set of File Manager Policy
- There are some standard elements common to ALL Data sets
- Data Sets are heterogeneous
...
So we DO NOT want to repeat the elements that both data sets have in common, so we introduce the idea of PARENT product types within the product-type-element-map.xml file. Whatever elements a PARENT product type contains are inherited by the children. In the Sample Policy below you see that Title and Year are declared ONCE but 2 child product types (Music and Video) can use those elements during ingestion.
Sample Policy Overview
NOTE: Items that are default policy and listed in the first example have been replaced with CAS.DEFAULTS to save space.
...
- type=Video parent=GenericFile
- +Resolution
- +Chapters
- +Aspect_Ratio
- +Director
Building File Manager Policy with Advanced Inheritance and Organization to support Growth
- Core Elements will be Setup and Used
- Great Divergence of metadata between Data Sets
- Need to easily add new unknown Data Sets in the Future
- Separation of Concerns (more than one person can update policy)
- The filemanager.properties file within the ./etc/ directory will need to include paths to all policy files, or the file manager will not pick them up.
- Virtual Product Types will be used
VIRTUAL vs. STANDARD Product Types
Before diving into this example we should take a moment to explain the difference between a VIRTUAL and STANDARD Product Type.
Virtual Product Types are only used for grouping and organizing metadata elements to support inheritance. These Product Types ARE NOT declared within a product-types.xml and they DO NOT have a MetExtractor associated with them.
Standard Product Types on the other hand MUST BE declared in product-types.xml and MetExtractors can be associated.
...
The hardest part about getting data from 5 different vendors is making them agree on what else they will add to the catalog, and for that we will enable Vendor Specific Policy. Along with having 5 vendors in the mix we need to make sure they cannot edit the CORE elements, and that they cannot edit EACH OTHERS elements, so we need to also talk about WHERE on disk the *.xml files will be stored and organized.
Sample Policy Overview
NOTE: Again to keep the example brief DEFAULTS will be used to represent large blocks of content that can be seen in previous examples. We will also include a FILEPATH above each filename to demonstrate WHERE on disk the files should be stored. You can store them where ever you like, this is just a suggestion of how to organize the policy files.
CORE POLICY
/usr/local/filemanageroodt/filemgr/policy/core
product-types.xml
- GenericFile
...
- type=Images parent=GenericFile <<<Images is a VIRTUAL Product Type
- IMAGE.DEFAULTS
VENDORX POLICY
/usr/local/filemanageroodt/filemgr/policy/vendorx
product-types.xml
...