Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Warning

This API can evolve in future releases and hence it is relevant to the state of the API in 1.99.5

 

Background

Sqoop2 supports a persistent store for the sqoop entities such as the Configurables ( Connector and Driver) , Configs exposed by the Connectors, Jobs and Jobruns etc. The persistent store is commonly referred to as the repository. We also expose Rest APIs and shell commands to perform CRUD operations on the sqoop entities such as connectors and drivers, connector configs related to link and job information, sqoop job and its configs. Thus the persistent store comes handy in keeping a history of the sqoop entity objects created and updated over time. In order to access the persistent store with ease, we also expose a simple java based repository API that different data stores can implement to store the sqoop entity objects.

...

The rest of the document will focus on the main public facing entities and  repository APIs 

 

Sqoop Entities

EntityModelRelationshipDescription
ConfigurableConfigurable.java Top Level Entity

 ConfigurableType Supported as of 1.99.5

CONNECTORMConnector.java
  • associated with Configurable
  • has 1-n config objects
is a type of configurable
DRIVERMDriver.java
  • associated with Configurable
  • has 1-n config objects
is a type of configurable  

 

...

CONFIG  Top Level Entity 
INPUT  Top Level Entity

Input Type supported as of 1.99.5

JOB_INPUT  Basically represents the values for the Job Type Inputs
LINK_INPUT  Basically represents the values for the Link Type Inputs
LINK has 1-n configs-inputs objects
 
JOB 

has 1-n configs-inputs objects

has 1-n submissions

 
Submission