Apache Atlas has a type system that can be used to build out specific structures for storing different types of metadata entities and the relationships between them. These metadata types are defined either using JSON files that are loaded into Atlas or through calls to the Types API.
Within the type system are four major elements that are defined in the JSON model files.
Element Description | Example |
---|---|
{ "name": "group", "typeName": "file_action", "cardinality": "SINGLE", "isIndexable": true, "isOptional": false, "isUnique": false }, | { "name": "file_action", "typeVersion": "1.0", "elementDefs": [ { "ordinal": 0, "value": "NONE" }, { "ordinal": 1, "value": "EXECUTE" }, { "ordinal": 2, "value": "WRITE" }, { "ordinal": 3, "value": "WRITE_EXECUTE" }, { "ordinal": 4, "value": "READ" }, { "ordinal": 5, "value": "READ_EXECUTE" }, { "ordinal": 6, "value": "READ_WRITE" }, { "ordinal": 7, "value": "ALL" } ] } |
{ "name": "sortCols", "typeName": "array<hive_order>", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false }, Structs are stored as their own vertex in the graph, linked to their entity. | { "name": "hive_order", "typeVersion": "1.0", "attributeDefs": [ { "name": "order", "typeName": "int", "cardinality": "SINGLE", "isIndexable": false, "isOptional": false, "isUnique": false }, { "name": "col", "typeName": "string", "cardinality": "SINGLE", "isIndexable": false, "isOptional": false, "isUnique": false } ] }, |
| <none currently available> |
| { "name": "Process", "superTypes": ["Referenceable", "Asset"], "typeVersion": "1.0", "attributeDefs": [ { "name": "inputs", "typeName": "array<DataSet>", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false }, { "name": "outputs", "typeName": "array<DataSet>", "cardinality": "SINGLE", "isIndexable": false, "isOptional": true, "isUnique": false } ] } |
Note:
seeks to add a 5th type for relationships called "relationshipDef". This will allow independent relationships to be added to the metadata.Prebuilt Models
The Atlas build includes some pre-built types that are located in addons/model. This directory includes a number of JSON files. These JSON files are loaded in numerical order. Once the files in addons/model are added then the files in addons/model/patches are loaded, again in numerical order.
Note: Building out the Apache Atlas Typesystem.
seeks to add a set of metadata types that will support a comprehensive open metadata and governance capability based on Apache Atlas. For more information see