You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Unable to render Jira issues macro, execution error.

 

The open connector framework governs access to data and metadata through standard mechanisms whilst preserving (where possible) the native data access APIs.

Accessing data APIs requires knowledge of the network address of the API for the data store, plus additional parameters such as userId and password.  These details are normally hard-coded in the calling application code which creates both a maintenance and security issue.  The OCF acts as a secure factory for connectors to data stores.  The application supplies the name of the connection it needs and assuming it is authorized, the OCF returns the connector.

Each connector has 2 APIs.  The first gives access to the data in the data store (the asset) and the second gives access to the metadata about the asset.  This means that applications have easy access to the metadata about the data assets they are accessing.

 

Inside the open connector framework

Figure 1 below shows a schematic of the core objects of the open connector framework.

 


 

  1. An application requests a connection to the data store by calling the Connector Broker.
  2. The connector broker looks up the the connection details in the Apache Atlas Metadata Repository.
  3. The connection details identifies the Connector Provider and the parameters it needs to create a Connector. 
  4. A connector is a java object. It is returned to the application by the connector broker
  5. The application is able to access both data and metadata through the connector.
  6. The connector extracts data from the data store.
  7. The connector extracts metadata from the Apache Atlas metadata repository.  It knows which asset metadata to return because it is linked to the connection details in Atlas.
Figure 1: Open Connector Framework - Overview of Operation

 

The OCF provides:

  • the implementation of the connector broker - there is both a Java and a RESTful API for the Connector Broker
  • the metadata model for the connector directory and connection metadata in Apache Atlas metadata repository
  • the APIs to enable the metadata to be maintained and accessed:
    • Connector Directory open metadata access service API - for managing connection metadata and organizing them into connector directories for convenience and access management
    • Asset open metadata access service API - for accessing an maintaining metadata about an asset.

In addition Apache Atlas will provide support for JDBC connectors and open metadata repositories (such as Apache Atlas).

Other vendors or open source projects may supply connector providers what are able to create connectors for different types of data assets.  

 


  • No labels