Sub-Systems
- Configuration: Manage system configuration
- Repository Manager:** Create Repository Schema** Create or edit connectors** Create or edit Connections
- Create or edit Jobs
- Connector Manager** Verify connectors** Register new connectors** Edit connectors (enable/disable)
- Job Manager:** Submit new jobs to MR** Get Progress of Jobs from MR** Kill specific job when requested
Initialization
- Configuration** Initialize configuration system
- Initialize logging infrastructure
- Repository Manager** Create repository Schema
- Verify if the schema is accurate
- Connector Manager** Register connectors
- Verify that all connector information is accurate
- Job Manager** Verify the status of jobs that were running at the time of last shutdown
Metadata
Component |
Resource |
Metadata |
Connector |
sc2, sc3 |
- Information needed to create connection |
Connection |
sx2, sx3 |
- Which connector backs this connection |
Job |
sj2, sj3 |
- Which connection the job is associated with |
Connector Metadata
- Information needed by the connector to create a connection
- Information needed by the connector to create a job, given a connection
Resources
Green resources are currently implemented in current (2012-10-17) version of sqoop2.
Id |
Resource |
Method |
Action/Parameters |
sv1 |
/version |
GET |
Return a list of supported protocol versions. Current list includes “1” only. |
sc1 |
/v1/connector |
GET |
Return a brief list of available connectors. |
sc2 |
/v1/connector/all |
GET |
Return a list of connectors with each connector’s metadata. |
sc3 |
/v1/connector/cid |
GET |
Return the metadata of the connector identified by cid |
sx1 |
/v1/connection |
GET |
Return a brief list of available connections. |
sx2 |
/v1/connection/all |
GET |
Return a list of available connections with metadata about each. |
sx3 |
/v1/connection/xid |
GET |
Return the metadata about connection identified by xid |
sx4 |
/v1/connection |
POST |
Create new connection. |
sx6 |
/v1/connection/xid |
PUT |
Update connection identified by xid. |
sx5 |
/v1/connection/xid |
DELETE |
Delete connection identified by xid. |
sj1 |
/v1/job |
GET |
Return a brief list of available jobs. |
sj2 |
/v1/job/all |
GET |
Return a list of available jobs with their configuration metadata. |
sj3 |
/v1/job/jid |
GET |
Return the metadata about job identified by jid |
sj4 |
/v1/job |
POST |
Create new job. |
sj5 |
/v1/job/jid |
PUT |
Update job identified by jid. |
sj6 |
/v1/job/jid |
DELETE |
Delete job identified by jid. |
ss1 |
/v1/status |
GET |
List of all jobs that are running |
ss2 |
/v1/status/all |
GET |
List of all jobs that are running with details about each job’s progress |
ss3 |
/v1/status/jid |
GET |
Details of th job identified by jid |
st1 |
/v1/jobcontrol |
POST |
Starts/Stop/Kill a job. “JOBID” → Sqoop specific Job ID |
sn1 |
/v1/notification |
POST |
Notification support for job progress |