Versions Compared

Old Version 4

changes.mady.by.user Veena Basavaraj

Saved on

compared with

New Version 5

changes.mady.by.user Veena Basavaraj

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Code Block
<!-- Listeners -->
   <listener>
    <listener-class>org.apache.sqoop.server.ServerInitializer</listener-class>
  </listener>
 

 

Sqoop Server

  • The sqoop server is represented by the java class SqoopServer.java
  • SqoopServer.initialize() is the called from the ServerInitiaizer
  • SqoopServer.destroy() is called when the tomcat server is shutdown

Sqoop Servlets

 
  • They receive the requests from the web or the Sqoop Client and process the request.
The
Code Block

  <!-- Version servlet -->
  <servlet>
    <servlet-name>VersionServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.VersionServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>VersionServlet</servlet-name>
    <url-pattern>/version</url-pattern>
  </servlet-mapping>
   <!-- Generic Configurable servlet -->
  <servlet>
    <servlet-name>v1.ConfigurableServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.ConfigurableServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.ConfigurableServlet</servlet-name>
    <url-pattern>/v1/configurable/*</url-pattern>
  </servlet-mapping>
  <!-- Connector servlet -->
  <servlet>
    <servlet-name>v1.ConnectorServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.ConnectorServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.ConnectorServlet</servlet-name>
    <url-pattern>/v1/connector/*</url-pattern>
  </servlet-mapping>
  <!-- Connectors servlet -->
  <servlet>
    <servlet-name>v1.ConnectorsServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.ConnectorServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.ConnectorsServlet</servlet-name>
    <url-pattern>/v1/connectors/*</url-pattern>
  </servlet-mapping>
  <!-- Driver servlet -->
  <servlet>
    <servlet-name>v1.DriverServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.DriverServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.DriverServlet</servlet-name>
    <url-pattern>/v1/driver/*</url-pattern>
  </servlet-mapping>
  <!-- Link servlet -->
  <servlet>
    <servlet-name>v1.LinkServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.LinkServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.LinkServlet</servlet-name>
    <url-pattern>/v1/link/*</url-pattern>
  </servlet-mapping>
 <!-- Links servlet -->
  <servlet>
    <servlet-name>v1.LinksServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.LinksServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.LinksServlet</servlet-name>
    <url-pattern>/v1/links/*</url-pattern>
  </servlet-mapping>
  <!-- Job servlet -->
  <servlet>
    <servlet-name>v1.JobServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.JobServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.JobServlet</servlet-name>
    <url-pattern>/v1/job/*</url-pattern>
  </servlet-mapping>
  <!-- Jobs servlet -->
  <servlet>
    <servlet-name>v1.JobsServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.JobsServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.JobsServlet</servlet-name>
    <url-pattern>/v1/jobs/*</url-pattern>
  </servlet-mapping>
  <!-- Submissions servlet -->
  <servlet>
    <servlet-name>v1.SubmissionsServlet</servlet-name>
    <servlet-class>org.apache.sqoop.server.v1.SubmissionsServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>v1.SubmissionsServlet</servlet-name>
    <url-pattern>/v1/submissions/*</url-pattern>
  </servlet-mapping>

</web-app>
 
Sqoop Request Handlers
 
Each Sqoop Servlet has its corresponding handler class that handles the request for that servlet. It then internally calls the internal sqoop core/ common code.
 Code Block
public interface RequestHandler {
  static final String CONNECTOR_NAME_QUERY_PARAM = "cname";
  static final String JOB_NAME_QUERY_PARAM = "jname";
  JsonBean handleEvent(RequestContext ctx);
}
public class ConnectorRequestHandler implements RequestHandler {
...
}
All the rest APIs supported via the sqoop-server are documented here: http://sqoop.apache.org/docs/1.99.4/RESTAPI.html#id1
Sqoop Client
  • Sqoop Client is represented by the java class SqoopClient.java
  • It has wrapper ResourceRequest classes for each sqoop entity, they encapsulate the request/postBody parameters to be sent in the request. Refer to Sqoop 2 (1.99.4) Entity Nomenclature and Relationships for more details on the supported Sqoop Entities.
  • It used the bare bones HttpURLConnection object to make requests to the Sqoop-server.
     Code Block
    HttpURLConnection conn = new DelegationTokenAuthenticatedURL().openConnection(url, authToken);
     Note
     SqoopClient used to use jersey REST client for making tomcat requests. Recently it was switched to Hadoop-auth/SPENGO for adding Kerberos support that are documented here
    https://cwiki.apache.org/confluence/display/SQOOP/Security+Guide+On+Sqoop+2