...

...

Overview

The Open Connector Framework (OCF) governs access to data assets and metadata through standard mechanisms whilst preserving (where possible) the native data asset access APIs.  It provides interfaces and factories for named connectors (called OCF Connectors) that access distributed data resources.   These data resources may be data stores (databases, files etcetera) or APIs to application data, transformations and analytical functions.

Accessing data through an APIs requires knowledge of the network address of the API for the data store, plus additional parameters such as userId and password.  These details are normally hard-coded in the calling application code which creates both a maintenance and security issue.  The OCF acts as a secure factory for connectors to data stores.  The application supplies the name of the connection it needs and assuming it is authorized, the OCF returns the connector.

...

• There are many existing connectors and connector framework in the industry today.  It is important that these connectors can be incorporated into the OCF.  Thus the OCF includes placeholders for adapters to external connector providers and connectors.
• Application developers will only adopt a connector framework if it is easy to use.  Thus the connector interfaces allow for the use of native data APIs to minimize the effort an application developer has to take in order to use the OCF connectors.
• Governance enforcement is a complex topic, typically managed externally to the application development team.   As a result, a separate framework called the Governance Action Framework (GAF) manages governance enforcement.  The role of the OCF is to bridge from the data resource access requests to the GAF.
• Access to the metadata about a connector and its associated data resource should benefit from the breadth of metadata about the data resource in Apache Atlasthe open metadata repositories.  Thus the OCF is dependent on the Open Metadata Access Services (OMASs) from Apache Atlas.

Key Concepts

...

Connection

The connection is a metadata entity that defines the set of parameters needed to access a specific data resource.  Each connection has a unique name.  An application can request a connector instance from the OCF using the name of a connection.  (See model

Connector Instance

A connector instance is a java object that implements the Connector API.   It provides access to a data resource, along with its related metadata stored in Apache Atlas.  The connector instance is responsible for calling the governance action framework when it is initialized and before and after every access request to the data resource.

...

Connector Directory OMAS

The Connector Directory OMAS is one of the Open Metadata Access Services (OMAS) that manages the configuration for connections and connector directories.

...

Figure 1 below shows the interaction of the core objects of the open connector framework.  The numbers on the diagram for Figure 1 show the order of execution.  The notes below describe this interaction.

...

• the metadata model for the connector directory and connection metadata in Apache Atlas metadata repository - see Area 1 2 - Assets and Connectors
• the APIs to enable the metadata to be maintained and accessed:
  • ConnectorDirectoryOMAS open metadata access service Connector Directory Open Metadata Access Service (OMAS) API - for managing connection metadata and organizing them into connector directories for convenience and access management
  • ConnectedAssetOMAS open metadata access service Connected Asset OMAS API - for accessing an maintaining metadata about an asset.

...

Once the OCF is in place Apache Atlas will provide support for JDBC connectors (see Virtual Data Connector (VDC)) and open metadata repositories (see Open Metadata Repository Services (OMRS)  OMRS Connectors).

Other vendors or open source projects may supply connector providers what are able to create connectors for different types of data assets.  

...