THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
This document outlines how to manually build and setup a Sqoop2 installation. It is expected that this setup will eventually be replaced by the assembly setup of Sqoop2 which will automate these steps.
| Table of Contents | ||||
|---|---|---|---|---|
|
Building Sqoop2
Checkout sources:
| No Format |
|---|
$ svn co https://svn.apache.org/repos/asf/sqoop/branches/sqoop2
|
And build them by doing the following:
| No Format |
|---|
$ cd sqoop2
$ mvn install
|
Setup Install Structure
Create another directory outside of Sqoop2 workspace where you will install the system.
| No Format |
|---|
$ cd ..
$ mkdir s2test
$ cd s2test
|
Within this directory, create subdirectories as noted below
| No Format |
|---|
$ mkdir bin conf logs repository
$ ls
$ ls -l
total 0
drwxr-xr-x 2 arvind staff 68 May 9 00:26 bin
drwxr-xr-x 2 arvind staff 68 May 9 00:26 conf
drwxr-xr-x 2 arvind staff 68 May 9 00:26 logs
drwxr-xr-x 2 arvind staff 68 May 9 00:26 repository
$
|
Add Tomcat Server
Download Tomcat version 7.0.X to a convenient location, and untar it in the s2test directory and rename the expanded directory to server.
| No Format |
|---|
$ wget http://apache.mirrors.hoobly.com/tomcat/tomcat-7/v7.0.27/bin/apache-tomcat-7.0.27.tar.gz
...
$ tar zxvf apache-tomcat-7.0.27.tar.gz
...
$ rm apache-tomcat-7.0.27.tar.gz
$ mv apache-tomcat-7.0.27 server
|
Now configure the Tomcat server to setup the configuration directory for Sqoop2. This can be done by creating a file called setenv.sh under server/bin directory. This file should contain the setting for Sqoop2 configuratiohn directory as follows:
| No Format |
|---|
$ # Working directory is s2test created earlier
$ echo "export JAVA_OPTS=-Dsqoop.config.dir=/path/to/directory/s2test/conf" >> server/bin/setenv.sh
$ chmod +x server/bin/setenv.sh
|
Setup Configuration
Under the s2test/conf directory, setup the bootstrap configuration as follows:
| No Format |
|---|
# File s2test/conf/sqoop_bootstrap.properties
qoop.config.provider=org.apache.sqoop.core.PropertiesConfigurationProvider
|
Now setup the main configuration as follows:
| No Format |
|---|
# File s2test/conf/sqoop.properties
# Log4J system
org.apache.sqoop.log4j.appender.file=org.apache.log4j.RollingFileAppender
org.apache.sqoop.log4j.appender.file.File=/path/to/s2test/logs/sqoop.log
org.apache.sqoop.log4j.appender.file.MaxFileSize=25MB
org.apache.sqoop.log4j.appender.file.MaxBackupIndex=5
org.apache.sqoop.log4j.appender.file.layout=org.apache.log4j.PatternLayout
org.apache.sqoop.log4j.appender.file.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} [%l] %m%n
org.apache.sqoop.log4j.debug=true
org.apache.sqoop.log4j.rootCategory=WARN, file
org.apache.sqoop.log4j.category.org.apache.sqoop=DEBUG
org.apache.sqoop.log4j.category.org.apache.derby=INFO
# Repository
org.apache.sqoop.repository.provider=org.apache.sqoop.repository.JdbcRepositoryProvider
org.apache.sqoop.repository.jdbc.handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler
org.apache.sqoop.repository.jdbc.transaction.isolation=READ_COMMITTED
org.apache.sqoop.repository.jdbc.maximum.connections=10
org.apache.sqoop.repository.jdbc.url=jdbc:derby:/path/to/s2test/repository/db;create=true
org.apache.sqoop.repository.jdbc.create.schema=true
org.apache.sqoop.repository.jdbc.driver=org.apache.derby.jdbc.EmbeddedDriver
org.apache.sqoop.repository.jdbc.user=sa
org.apache.sqoop.repository.jdbc.password=
org.apache.sqoop.repository.sysprop.derby.stream.error.file=/path/to/s2test/logs/derbyrepo.log
|