Building the Software

Describes the steps required to build and run Apache Trafodion.

Supported Platforms

Red Hat or Centos 6.x (6.4 or later) versions are supported as development and production platforms.

Prerequisites

You need to install the following packages before you can install Apache Trafodion.

sudo yum install epel-release

sudo yum install alsa-lib-devel ant ant-nodeps boost-devel cmake \
         device-mapper-multipath dhcp flex gcc-c++ gd git glibc-devel \
         glibc-devel.i686 graphviz-perl gzip java-1.7.0-openjdk-devel \
         libX11-devel libXau-devel libaio-devel \
         libcurl-devel libibcm.i686 libibumad-devel libibumad-devel.i686 \
         libiodbc libiodbc-devel librdmacm-devel librdmacm-devel.i686 \
         libxml2-devel log4cxx log4cxx-devel lua-devel lzo-minilzo \
         net-snmp-devel net-snmp-perl openldap-clients openldap-devel \
         openldap-devel.i686 openmotif openssl-devel openssl-devel.i686 \
         openssl-static perl-Config-IniFiles perl-Config-Tiny \
         perl-DBD-SQLite perl-Expect perl-IO-Tty perl-Math-Calc-Units \
         perl-Params-Validate perl-Parse-RecDescent perl-TermReadKey \
         perl-Time-HiRes protobuf-compiler protobuf-devel python-qpid \
         python-qpid-qmf qpid-cpp-client \
         qpid-cpp-client-ssl qpid-cpp-server qpid-cpp-server-ssl \
         qpid-qmf qpid-tools readline-devel saslwrapper sqlite-devel \
         unixODBC unixODBC-devel uuid-perl wget xerces-c-devel xinetd

Once installed, check the following.

Java Version

The Java version must be 1.7.x. Check as following:

$ java -version
java version "1.7.0_85"
OpenJDK Runtime Environment (rhel-2.6.1.3.el6_6-x86_64 u85-b01)
OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

Ensure that the Java environment exists and points to your JDK installation.

$ echo $JAVA_HOME
 
$ which java
/usr/bin/java
$ export JAVA_HOME=/usr/bin
$

You should export JAVA_HOME in your .bashrc or .profile file.

Verify Trafodion Download

Verify that the Trafodion source has been either:

Downloaded and unpackaged.
Cloned from github.

If not, please do so now. Refer to Contributor Workflow - Code/Docs.

Install Required Build Tools

Refer to Required Build Tools for instructions.

Verify System Limits

Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.

Recommended System Limits

ulimit –a
core file size             (blocks, -c) 1000000
data seg size              (kbytes, -d) unlimited
scheduling priority        (-e) 0
file size                  (blocks, -f) unlimited
pending signals            (-i) 515196
max locked memory          (kbytes, -l) 49595556
max memory size            (kbytes, -m) unlimited
open files                 (-n) 32000
pipe size                  (512 bytes, -p) 8
POSIX message queues       (bytes, -q) 819200
real-time priority         (-r) 0
stack size                 (kbytes, -s) 10240
cpu time                   (seconds, -t) unlimited
max user processes         (-u) 267263
virtual memory             (kbytes, -v) unlimited
file locks                 (-x) unlimited

Please refer to this article for information on how you change system limits.

Build Trafodion

Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.

cd <download directory>/incubator-trafodion
export TOOLSDIR=<tools installation directory>
source ./env.sh

If you do not set up TOOLSDIR before sourcing in env.sh, then tools location defaults to /opt/home/tools. You may want to edit your .bashrc or .profile file to always export TOOLSDIR.

Build using one of the following options:

Command	What It Builds
make all	Trafodion, DCS, and REST.
make package	Trafodion, DCS, REST, and Client Drivers.
make package-all	Trafodion, DCS, REST, Client Drivers, and tests for all components.

Verify the build:

Verify Build

$ sqvers -u
MY_SQROOT=/home/gtapper/incubator-trafodion/core/sqf
who@host=gtapper@box01
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.el6.x86_64
redhat=6.6
NO patches
Most common Apache_Trafodion Release 2.0.0 (Build debug [gtapper], branch 392b99b-trafweb, date 06Nov15)
UTT count is 1
[8]     Release 2.0.0 (Build debug [gtapper], branch 392b99b-trafweb, date 06Nov15)
          export/lib/hbase-trx-cdh5_3-2.0.0.jar
          export/lib/hbase-trx-hbase_98_4-2.0.0.jar
          export/lib/hbase-trx-hdp2_2-2.0.0.jar
          export/lib/hbase-trx-mapr4_0-2.0.0.jar
          export/lib/sqmanvers.jar
          export/lib/trafodion-dtm-2.0.0.jar
          export/lib/trafodion-sql-2.0.0.jar
          export/lib/trafodion-utility-2.0.0.jar
$

The output should show that eight jar files exist.

Setup Test Environment

You should test your installation using:

A local Hadoop environment
Trafodion installation

You're installation approach depends on whether you already have installed Hadoop.

If Hadoop is Already Installed

Build binary tar files and then install Trafodion following instructions described in Installation.

Build Binary Files

cd <download directory>/incubator-trafodion
make package

The binary tar files will be created in <download directory>/incubator-trafodion/distribution directory.

Install a Local Hadoop Environment

The install_local_hadoop script downloads compatible versions of Hadoop, HBase, Hive, and MySQL. Then, it starts Trafodion.

Time Saver

install_local_hadoop downloads Hadoop, HBase, Hive, and MySql jar files from the internet. To avoid this overhead, you can download the required files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to this directory.

Prerequisite

You need to ensure that you've set up passwordless authentication.

passwordless ssh Not Correctly Setup

ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 88:f7:90:fe:35:76:10:92:cf:6d:c4:60:ca:b9:43:9f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
gtapper@localhost's password:
Last login: Fri Nov  6 22:44:00 2015 from 192.168.1.9

Setup passwordless ssh

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Usage

Command	What It Does
install_local_hadoop	Uses default ports for all services.
install_local_hadoop -p fromDisplay	Start Hadoop with a port number range determined from the DISPLAY environment variable.
install_local_hadoop -p rand	Start with any random port number range between 9000 and 49000.
install_local_hadoop -p <port >	Start with the specified port number.

Default ports.

Port Variable	Default Value
MY_DCS_MASTER_PORT	23400
MY_DCS_MASTER_INFO_PORT	24400
MY_REST_SERVER_PORT	4200
MY_REST_SERVER_SECURE_PORT	4201

Sample Procedure

If Starting a New ssh Session

# Ensure that the Trafodion environmental variables have been loaded.
cd <download directory>/incubator-trafodion
source ./env.sh

Install the Hadoop software.

cd $MY_SQROOT/sql/scripts
install_local_hadoop
./install_traf_components

Verify installation.

$ swstatus
5 java servers and 2 mysqld processes are running
713 NameNode
1003 SecondaryNameNode
838 DataNode
1173 ResourceManager
1298 NodeManager
$

Five java servers and two mysqld processes should be running.

Manage Hadoop Environment

Use the following commands to manage the Hadoop environment.

Command	Usage
swstartall	Start the complete Hadoop environment.
swstopall	Stops the complete Hadoop environment.
swstatus	Checks the status of the Hadoop environment.
swuninstall_local_hadoop	Removes the Hadoop installation.

Run Trafodion

This section describes how to start Trafodion and run operations.

Each Time New Source is Downloaded

You need to do the following each time you download new source code.

cd <download directory>/apache-trafodion-1.3.0-incubating
source ./env.sh
cd $MY_SQROOT/etc
# delete ms.env, if it exists
rm ms.env
cd $MY_SQROOT/sql/scripts
sqgen

Start Trafodion

Do the following to start the Trafodion environment.

cd $MY_SQROOT/sql/scripts
sqstart
sqcheck

Management Scripts

Component	Start	Stop	Status
All of Trafodion	sqstart	sqstop	sqcheck
DCS (Database Connectivity Services)	dcstart	dcsstop	dcscheck
REST Server	reststart	reststop
LOB Server	lobstart	lobstop
RMS Server	rmsstart	rmsstop	rmscheck

Create Trafodion Metadata