You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

This document outlines how to manually build and setup a Sqoop2 installation. It is expected that this setup will eventually be replaced by the assembly setup of Sqoop2 which will automate these steps.

Building Sqoop2

Checkout sources:

$ svn co https://svn.apache.org/repos/asf/sqoop/branches/sqoop2

And build them by doing the following:

$ cd sqoop2
$ mvn install

Setup Install Structure

Create another directory outside of Sqoop2 workspace where you will install the system. 

$ cd ..
$ mkdir s2test
$ cd s2test

Within this directory, create subdirectories as noted below

$ mkdir bin conf logs repository 
$ ls
$ ls -l
total 0
drwxr-xr-x 2 arvind staff 68 May  9 00:26 bin
drwxr-xr-x 2 arvind staff 68 May  9 00:26 conf
drwxr-xr-x 2 arvind staff 68 May  9 00:26 logs
drwxr-xr-x 2 arvind staff 68 May  9 00:26 repository
$

Add Tomcat Server

Download Tomcat version 7.0.X to a convenient location, and untar it in the s2test directory and rename the expanded directory to server.

$ wget http://apache.mirrors.hoobly.com/tomcat/tomcat-7/v7.0.27/bin/apache-tomcat-7.0.27.tar.gz
...
$ tar zxvf apache-tomcat-7.0.27.tar.gz
...
$ rm apache-tomcat-7.0.27.tar.gz
$ mv apache-tomcat-7.0.27 server

Now configure the Tomcat server to setup the configuration directory for Sqoop2. This can be done by creating a file called setenv.sh under server/bin directory. This file should contain the setting for Sqoop2 configuratiohn directory as follows:

$ # Working directory is s2test created earlier
$ echo "export JAVA_OPTS=-Dsqoop.config.dir=/path/to/directory/s2test/conf" >> server/bin/setenv.sh
$ chmod +x server/bin/setenv.sh

Setup Configuration

Under the s2test/conf directory, setup the bootstrap configuration as follows:

# File s2test/conf/sqoop_bootstrap.properties
qoop.config.provider=org.apache.sqoop.core.PropertiesConfigurationProvider

Now setup the main configuration as follows:

# File s2test/conf/sqoop.properties
# Log4J system
org.apache.sqoop.log4j.appender.file=org.apache.log4j.RollingFileAppender
org.apache.sqoop.log4j.appender.file.File=/path/to/s2test/logs/sqoop.log
org.apache.sqoop.log4j.appender.file.MaxFileSize=25MB
org.apache.sqoop.log4j.appender.file.MaxBackupIndex=5
org.apache.sqoop.log4j.appender.file.layout=org.apache.log4j.PatternLayout
org.apache.sqoop.log4j.appender.file.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} [%l] %m%n

org.apache.sqoop.log4j.debug=true
org.apache.sqoop.log4j.rootCategory=WARN, file
org.apache.sqoop.log4j.category.org.apache.sqoop=DEBUG
org.apache.sqoop.log4j.category.org.apache.derby=INFO

# Repository
org.apache.sqoop.repository.provider=org.apache.sqoop.repository.JdbcRepositoryProvider
org.apache.sqoop.repository.jdbc.handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler
org.apache.sqoop.repository.jdbc.transaction.isolation=READ_COMMITTED
org.apache.sqoop.repository.jdbc.maximum.connections=10
org.apache.sqoop.repository.jdbc.url=jdbc:derby:/path/to/s2test/repository/db;create=true
org.apache.sqoop.repository.jdbc.create.schema=true
org.apache.sqoop.repository.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
org.apache.sqoop.repository.jdbc.user=sa
org.apache.sqoop.repository.jdbc.password=
org.apache.sqoop.repository.sysprop.derby.stream.error.file=/path/to/s2test/logs/derbyrepo.log
  • No labels