Page History

...

Install Cloudera or Horton works Hadoop distribution. In situations where you do not have the hadoop distributions already available you can use the optional sandbox install steps install local hadoop script as described below in . See section "Sandbox Set up Hadoop Installdistribution "
Java 1.7.x or greater must be installed. Ensure JAVA_HOME environment environment variable
is
exists and set
and points
to your JDK installation.
Download, build and install additional development tools via Additional Build Tools
Install the following packages via yum install <package>

...

sudo rpm -Uvh epelrelease-6-8.noarch.rpm

Compiling and Configuring Trafodion

To build and compile Trafodion

...

Set up Hadoop distribution (Install Hadoop, Hbase, Hive to local workspace)

You can use single node standalone Apache install OR use Trafodion supplied ‘install_local_hadoop’ script customized to install Cloudera distribution for a single node. When using Trafodion supplied Hadoop install script you can do the following:

Make sure you have set up password less authentication. Basically you should be able to "ssh localhost" without having to enter a password

Download latest Apache Trafodion source from Apache Incubator https://github.com/apache/incubator-trafodion
Using ssh, set Trafodion environment

...

cd incubator-trafodion

...

Configuring DCS and starting Trafodion

Open a new terminal via ssh
cd <apache-incubator-dir>/core/sqf

. ./sqenv.

; . ./env.sh
cd $MY_SQROOT/sql/scripts

sqgen
sqstart (Wait till all process is up and running)
sqcheck
sqlci > initialize trafodion;
Perform a quick sanity test of the install, by creating a sample table and querying it
1. sqlci
  >set schema trafodion.sch;
  >create table t (a int not null, primary key(a));
  >insert into t values (1), (3);
  >select * from t;
  >exit;
Edit $DCS_INSTALL_DIR/conf/dcs-site.xml and

Sandbox Hadoop install (Optional)

The instructions below describe steps to install and start sand boxed version of Hadoop using non-default ports

...

install_local_hadoop -p <port>

...

Execute the script ‘install_local_hadoop’

Note:

This script will download Hadoop and HBase jar files from the internet. To avoid this overhead for future executions of the script, you can save the downloaded files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to that directory.

...

The files to save are: $MY_SQROOT/sql/local_hadoop/*.tar.gz $MY_SQROOT/sql/local_hadoop/tpcds/tpcds_kit.zip.

...

swstartall

Hadoop Components (needs more work here...)

Install Hadoop, HBase and Hive to a your local workspace. Dependencies for release 0.9.x

Download the files to $HOME/tools folder. If 'tools' folder does not exist please create before issuing a wget command

wget http://archive-primary.cloudera.com/cdh5/cdh/5/hbase-0.98.1-cdh5.1.0.tar.gz

wget http://archive.apache.org/dist/hive/hive-0.13.1/apache-hive-0.13.1-bin.tar.gz

wget http://archive.apache.org/dist/hadoop/common/hadoop-2.4.0/hadoop-2.4.0-src.tar.gz

Untar the files in $HOME/tools folder

tar xzf apache-hive-0.13.1-bin.tar.gz $HOME/tools/apache-hive-0.13.1-bin

          tar xzf hbase-0.98.1-cdh5.1.0.tar.gz $HOME/tools/hbase-0.98.1-cdh5.1.0

tar xzf $HOME/tools/hadoop-2.4.0-src.tar.gz

          cd hadoop-2.4.0-src
          export JAVA_HOME=...           # path to 1.7.x JDK
          export HADOOP_PROTOC_PATH=...  # path to protobufs 2.5.0 protoc command
          mvn clean install package -Pdist -Pnative -Dtar -DskipTests \
             -Dtomcat.download.url=http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.36/bin/apache-tomcat-6.0.36.tar.gz

c. For Pre-installed Hadoop/HBase version, Update the HBase configuration and restart HBase.

...

‘install_local_hadoop –p rand' — will start with any random port number between 9000 and 49000

OR

‘install_local_hadoop –p < specify a port # >' — will start with port number specified

OR

‘install_local_hadoop' — will use default port numbers for all services

To start/stop/check Hadoop environment when using Trafodion supplied Hadoop install script, you can execute ‘swstartall’ , ‘swstopall’ and ‘swstatus’

For Hadoop installs that did not use Trafodion supplied Hadoop install script, please update HBase configuration as shown below and restart HBase

For hbase-site.xml:

<name>hbase.client.scanner.caching</name>

...

...

</property>

...

<name>hbase.client.scanner.timeout.period</name>

...

...

</property>

...

<name>hbase.coprocessor.region.classes</name>

...

<value>

org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,

...

org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,

...

org.apache.hadoop.hbase.coprocessor.AggregateImplementation

...

</value>

...

</property>

...

<name>hbase.hregion.impl</name>

...

<value>org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion</value>

...

</property>

...

For hbase-env.xml:

...

export HBASE_CLASSPATH=${HBASE_TRXDIR}/${HBASE_TRX_JAR}

...

To compile and configure Trafodion and its components

...

Set your

TOOLSDIR environment variable

...

to the

...

location of

...

Alternatively, use the install_local_hadoop script. See step 3 under the "Build" heading below.

NOTE: The hadoop release contains 32-bit libraries. You must build hadoop from source for 64-bit architecture, and not just download the release tar file. See: http://wiki.apache.org/hadoop/HowToContribute

Notes

...

install_local_hadoop -p <start-port>
<start-port> ... <start-port>+199 should be a range of unused ports.

...

 sqstop
 # edit source files
 cd $MY_SQROOT/..
 make all
 sqstart

...

 sqstop
 swstopall

To start it up later, use the following commands:

 swstartall
 sqstart

To check on the status, use these commands:

 sqcheck
 swstatus

...

the components installed via Additional Build Tools
If you have not already downloaded Apache Trafodion source now you can download it from here https://github.com/apache/incubator-trafodion
1. Using a new ssh session, set Trafodion environment
  cd incubator-trafodion
  . ./env.sh
  make all (Build Trafodion, DCS, REST) OR

make package (Build Trafodion, DCS, REST, Client drivers) OR

make package-all (Build Trafodion, DCS, REST, Client drivers and Tests for all components)

b. . cd $MY_SQROOT/sql/scripts

c. Execute script , ‘install_traf_components’. Based on the tar files available in the distribution folder this script will install the Trafodion components.

Note: All tar files will be created in ‘distribution’ folder located at the very top level (incubator-trafodion)

To install a custom Trafodion component you can set various environment variable to overwrite the default tar files found in distribution folder.

Environment variables supported by install_traf_component script are :

DCS_TAR — Specify the fully qualified path of DCS tar file

REST_TAR — Specify the fully qualified path of REST tar file

PHX_TAR — Specify the fully qualified path of Phoenix test tar file

CLIENT_TAR — Specify the fully qualified path of Trafodion client tar file

DCSTEST_TAR — Specify the fully qualified path of DCS tests tar file

Starting Trafodion and its components

Using a new ssh session,

cd incubator-trafodion; Execute . ./env.sh
cd $MY_SQROOT/sql/scripts
Execute the script, ‘sqgen’ and then start using the script ‘sqstart’

Note: In case of any issues and if there is a need to stop and restart a specific Trafodion component, you can use the component based start/stop scripts.

Component	Start script	Stop script
For all of Trafodion	sqstart	sqstop
For DCS (Database Connectivity Service)	dcsstart	dcsstop
For REST server	reststart	reststop
For LOB server	lobstart	lobstop
For RMS server	rmsstart	rmsstop

Checking the status of Trafodion and its components

There are several health check scripts that are available which will provide the status of Trafodion. They are :

sqcheck (For all of Trafodion)

dcscheck (For Database Connectivity Service)

rmscheck (For RMS Server)

Creating Trafodion metadata

Using a new ssh session,

1. cd incubator-trafodion; . ./env.sh
2. cd $MY_SQROOT/sql/scripts
3. Use sqlci (direct to sql engine) or trafci (uses DCS to connect to SQL engine)
4. Execute the sql initialization script via sqlci or trafci, ‘initialize trafodion’

Testing Trafodion

There are several helper scripts provided to run the tests for Trafodion components in your workspace. These scripts are generated based on the tar files that are made available during execution of install_traf_component script

swphoenix {t4 | t2 } — This script will run the Phoenix test using JDBC Type 4 driver or JDBC Type 2 driver

swjdbc — This script will run JDBC Type 4 tests

swpyodbc — This script will install Linux drivers run ODBC tests using Linux driver

Space shortcuts

Page tree

Versions Compared

Old Version 9

New Version 10

Key

Compiling and Configuring Trafodion

To build and compile Trafodion

Set up Hadoop distribution (Install Hadoop, Hbase, Hive to local workspace)

Configuring DCS and starting Trafodion

Sandbox Hadoop install (Optional)

Note:

Hadoop Components (needs more work here...)

To compile and configure Trafodion and its components

Notes

Starting Trafodion and its components

Checking the status of Trafodion and its components

Creating Trafodion metadata

Testing Trafodion