Page History

Warning

title	Obsolete Page

This page is being obsoleted and is replaced by: http://trafodion.apache.org/build.html

Panel

borderColor	gray
borderWidth	1
borderStyle	solid

Describes the steps required to build and run Apache Trafodion.

Supported Platforms

Red Hat or Centos 6.x (6.4 or later) versions are supported as development and production platforms.

...

Build prerequisites

You need to install the following packages before you can install Apache Trafodion.

...

Code Block

language	bash

$ java -version
java version "1.7.0_85"
OpenJDK Runtime Environment (rhel-2.6.1.3.el6_6-x86_64 u85-b01)
OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

Ensure

...

that

...

the

...

Java

...

environment

...

exists

...

and

...

points

...

to

...

your

...

JDK

...

installation.

...

By default Java is located in /usr/lib/java-<version>.

Code Block

language

Code Block

language	bash

$ echo $JAVA_HOME
 
$ which java
/usr/bin/java
$ export JAVA_HOME=/usr/bin
$/lib/jvm/java-1.7.0-openjdk.x86_64

Note
You should export JAVA_HOME in your .bashrc or .profile file.

...

If not, please do so now. Refer to

To download from github, refer to Contributor Contributor Workflow - Code/Docs.
Otherwise downloaded and untar the source tar file from Apache Trafodion Incubator release.

Install Required Build Tools

Trafodion requires a set of tools to be installed in order to build. Refer to Required Build Tools for instructions.

Verify System Limits

Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.

One of the tools that is built if it does not already exist is Maven. At this time, you should verify that Maven is part of your path. To verify that Maven is available do:

mvn --version.

If not found, then added it to your path:

PATH=$PATH:<tool installation directory>/apache-maven-3.3.3/bin

Note
You should add Maven to your PATH in your .bashrc or .profile file.

Build Trafodion

Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.

<Trafodion source directory> is source tree base for Trafodion.
<tools installation directory> is where Trafodion required tools are located. The following example assumes that you installed all the required tools in a single location. If you installed or used pre-installed tools in different directories, then you need to export the location of each tool as described Required Build Tools prior to sourcing in env.sh.

Code Block

language	text

cd <Trafodion source directory>
export TOOLSDIR=<tools installation directory>
source ./env.sh

Build a debug version of Trafodion using one of the following options:

Command	What It Builds
make all	Trafodion, DCS, and REST.
make package	Trafodion, DCS, REST, and Client Drivers.
make package-all	Trafodion, DCS, REST, Client Drivers, and tests for all components.

If the build fails, you might want to rerun the make step. Trafodion downloads many dependencies and sometimes one of the download operations fail. Rerunning the build generally works.

Verify the build:

Code Block

language	text
title	Verify Build

$ sqvers -u
MY_SQROOT=/home/centos/apache-trafodion-1.3.0-incubating/core/sqf
who@host=centos@mysystem
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.1.3.el6.x86_64
redhat=6.7
NO patches
Most common Apache_Trafodion Release 1.3.0 (Build debug [centos], branch -, date 06Nov15)
UTT count is 1
[6]     Release 1.3.0 (Build debug [centos], branch -, date 06Nov15)
          export/lib/hbase-trx-cdh5_3-1.3.0.jar

Code Block

language	text
title	Recommended System Limits

ulimit –a
core file size             (blocks, -c) 1000000
data seg size              (kbytes, -d) unlimited
scheduling priority        (-e) 0
file size                  (blocks, -f) unlimited
pending signals            (-i) 515196
max locked memory          (kbytes, -l) 49595556
max memory size            (kbytes, -m) unlimited
open files                 (-n) 32000
pipe size                  (512 bytes, -p) 8
POSIX message queues       (bytes, -q) 819200
real-time priority         (-r) 0
stack sizeexport/lib/hbase-trx-hbase_98_4-1.3.0.jar
               export/lib/hbase-trx-hdp2_2-1.3.0.jar
   (kbytes, -s) 10240
cpu time    export/lib/sqmanvers.jar
          export/lib/trafodion-dtm-1.3.0.jar
     (seconds, -t) unlimited
max user processes         (-u) 267263
virtual memory             (kbytes, -v) unlimited
file locks                 (-x) unlimited

Please refer to this article for information on how you change system limits.

Build Trafodion

Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.

 export/lib/trafodion-sql-1.3.0.jar

The output show several jar files. The number of files differs based on the version of Trafodion you downloaded.

Setup Test Environment

You should test your installation using:

Trafodion installation on a system that already has a compatible version of Hadoop installed
A local Hadoop environment created by the install_local_hadoop script

Your installation approach depends on whether you already have installed Hadoop.

Hadoop is Already Installed

Build binary tar files and then install Trafodion following instructions described in Installation.

...

Code Block

language	text
title	Build Binary Files

cd <Trafodion <downloadsource directory>/incubator-trafodion
export TOOLSDIR=<tools installation directory>
source ./env.sh

Note
If you do not set up TOOLSDIR before sourcing in env.sh, then tools location defaults to /opt/home/tools. You may want to edit your .bashrc or .profile file to always export TOOLSDIR.

Build using one of the following options:

Command	What It Builds
make all	Trafodion, DCS, and REST.
make package	Trafodion, DCS, REST, and Client Drivers.
make package-all	Trafodion, DCS, REST, Client Drivers, and tests for all components.

Verify the build:

make package

The binary tar files will be created in <Trafodion source directory>/distribution directory.

Install a Local Hadoop Environment

Local Hadoop prerequisites

Setup Passwordless SSH

Check to see if you have passwordless SSH setup.

Code Block

language	text
title	passwordless ssh setup correctly

ssh localhost
Last login: Fri Nov  6 22:44:00 2015 from 192.168.1.9

If passwordless SSH is not setup, please do so now. The following is an example of setting up passwordless SSH using id_rsa keys. You can choose the method that best represents your environment.

If you already have an existing set of ssh keys. Simply copy both the id_rsa.pub and id_rsa to your ~/.ssh directory.

Then, do the following:

Code Blockcode

language	text
title	Verify Build

$ sqvers -u

The output should show that seven jar files exist.

Prepare the Test Environment

You should test your installation using:

A local Hadoop environment
Trafodion installation

You're installation approach depends on whether you already have installed Hadoop.

If Hadoop is Already Installed

Build binary tar files and then install Trafodion following instructions described in Installation.

Code Block

language	text
title	Build Binary Files

cd <download directory>/incubator-trafodion
make package

The binary tar files will be created in <download directory>/incubator-trafodion/distribution directory.

...

Modify the ssh Environment

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 ..

If you need to create your keys first, then do the following:

Code Block

language	text
title	Setup passwordless ssh

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa.pub
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 ..

Verify System Limits

Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.

Code Block

language	text
title	Recommended System Limits

ulimit –a
core file size             (blocks, -c) 1000000
data seg size              (kbytes, -d) unlimited
scheduling priority        (-e) 0
file size                  (blocks, -f) unlimited
pending signals            (-i) 515196
max locked memory          (kbytes, -l) 49595556
max memory size            (kbytes, -m) unlimited
open files                 (-n) 32000
pipe size                  (512 bytes, -p) 8
POSIX message queues       (bytes, -q) 819200
real-time priority         (-r) 0
stack size                 (kbytes, -s) 10240
cpu time                   (seconds, -t) unlimited
max user processes         (-u) 267263
virtual memory             (kbytes, -v) unlimited
file locks                 (-x) unlimited

Please refer to this article for information on how you change system limits.

Run install_local_hadoop

The install_local_hadoop script downloads compatible versions of Hadoop, HBase, Hive, and MySQL. Then, it starts Trafodion.

...

title	Time Saver

...

Hadoop, HBase, Hive, and

...

MySQL. Then, it starts Trafodion.

Tip

title	Time Saver

install_local_hadoop downloads Hadoop, HBase, Hive, and MySql jar files from the internet. To avoid this overhead, you can download the required files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to this directory

Usage

Default ports.

Port Variable	Default Value
MY_DCS_MASTER_PORT	23400
MY_DCS_MASTER_INFO_PORT	24400
MY_REST_SERVER_PORT	4200
MY_REST_SERVER_SECURE_PORT	4201

...

.

<port > the specified port number.

Command	What It Does
install_local_hadoop	Uses default ports for all services.
install_local_hadoop -p fromDisplay	Start Hadoop with a port number range determined from the DISPLAY environment variable.	install_local_hadoop -p rand	Start with any random port number range between 9000 and 49000with a port number range determined from the DISPLAY environment variable.
install_local_hadoop -p	rand	Start with

Sample Procedure

Code Block

language	text
title	If Starting a New ssh Session

# Ensure that the Trafodion environmental variables have been loaded.
cd <download directory>/incubator-trafodion
source ./env.sh

You need to ensure that you've set up passwordless authentication.

any random port number range between 9000 and 49000.
install_local_hadoop -p <port >	Start with the specified port number.

For a list of ports that get configured and their default values, see Configure Ports for a Firewall.

Sample Procedure

Code Block

language	text
title	passwordless ssh Not Correctly Setup

ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 88:f7:90:fe:35:76:10:92:cf:6d:c4:60:ca:b9:43:9f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
gtapper@localhost's password:
Last login: Fri Nov  6 22:44:00 2015 from 192.168.1.9

Code Block

language	text
title	Setup passwordless ssh

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

If Starting a New ssh Session

# Ensure that the Trafodion environmental variables have been loaded.
cd <Trafodion source directory>
source ./env.sh

Install the Hadoop software.

Code Block

language	text

cd $MY_SQROOT/sql/scripts
install_local_hadoop
./install_traf_components

Verify installation.

Code Block

language	text

$ swstatus
6 java servers and 2 mysqld processes are running
713   NameNode
19513 HMaster
1003  SecondaryNameNode
838   DataNode
1173  ResourceManager
1298  NodeManager

Six java servers as shown above and two mysqld processes should be running.

Manage Hadoop Environment

Use the following commands to manage the Hadoop environment.

Command	Usage
swstartall	Start the complete Hadoop environment.
swstopall	Stops the complete Hadoop environment.
swstatus	Checks the status of the Hadoop environment.
swuninstall_local_hadoop	Removes the Hadoop installation.

Run Trafodion

This section describes how to start Trafodion and run operations.

Each Time New Source is Downloaded

You need to do the following each time you download new source codeInstall the Hadoop software.

Code Block

language	text

cd <Trafodion source directory>
source ./env.sh
cd $MY_SQROOT/sql/scripts
install_local_hadoop
./install_traf_components

Verify installation.

Code Block

language	text

swstatus

Six java servers and two mysqld processes should be running.

To start/stop/check Hadoop environment using Trafodion supplied scripts, you can execute ‘swstartall’ , ‘swstopall’ and ‘swstatus’, and if you need to remove the installation, execute the 'swuninstall_local_hadoop'.

Run Trafodion

This section describes how to start Trafodion and run operations.

Setup required each time source is downloaded

Start a new ssh session and set up environment:

    cd <download directory>/apache-trafodion-1.3.0-incubating
    source ./env.sh
    cd $MY_SQROOT/etc
    # delete ms.env, if it exists
    rm ms.env
    cd $MY_SQROOT/sql/scripts
    sqgen

Start up Trafodion

    cd $MY_SQROOT/sql/scripts
    sqstart
    sqcheck

Note: In case of any issues and if there is a need to stop and restart a specific Trafodion component, you can use the component based start/stop scripts.

Component	Start script	Stop script
For all of Trafodion	sqstart	sqstop
For DCS (Database Connectivity Service)	dcsstart	dcsstop
For REST server	reststart	reststop
For LOB server	lobstart	lobstop
For RMS server	rmsstart	rmsstop

Checking the status of Trafodion and its components

There are several health check scripts that are available which will provide the status of Trafodion. They are :

sqcheck (For all of Trafodion)
dcscheck (For Database Connectivity Service)
rmscheck (For RMS Server)

Create Trafodion metadata

If you started a new ssh session, be sure to

   cd <download directory>/apache-trafodion-1.3.0-incubating
   source ./env.sh

Trafodion is up and running, you can now start up a SQL command line interface and initialize Trafodion

   sqlci

Perform the following statements:

    initialize trafodion;
    exit

Validate

Test your setup by using "sqlci" or "trafci" (uses DCS to connect to the SQL engine):

    get schemas;
    create table table1 (a int);
    invoke table1;
    insert into table1 values (1), (2), (3), (4);
    select * from table1;
    exit;

etc
# delete ms.env, if it exists
rm ms.env
cd $MY_SQROOT/sql/scripts
sqgen

Start Trafodion

Do the following to start the Trafodion environment.

Code Block

language	text

cd $MY_SQROOT/sql/scripts
sqstart
sqcheck

Management Scripts

Component	Start	Stop	Status
All of Trafodion	sqstart	sqstop	sqcheck
DCS (Database Connectivity Services)	dcstart	dcsstop	dcscheck
REST Server	reststart	reststop
LOB Server	lobstart	lobstop
RMS Server	rmsstart	rmsstop	rmscheck

Create Trafodion Metadata

Code Block

language	text
title	If Starting a New ssh Session

# Ensure that the Trafodion environmental variables have been loaded.
cd <Trafodion source directory>
source ./env.sh

Assumption: Trafodion is up and running.

Use sqlci to create the Trafodion metadata.

Code Block

language	text

$ sqlci
>> initialize trafodion;
.
.
.
>> exit;
$

Validate Your Installation

You can use sqlci or trafci (connects via DCS) to validate your installation.

Code Block

language	sql
title	sqlci or trafci

get schemas;
create table table1 (a int);
invoke table1;
insert into table1 values (1), (2), (3), (4);
select * from table1;
exit;

Assuming no errors, your installation has been successful. You can start working on your modifications.

Troubleshooting Notes

If you are not able to start up the environment or if there are problems running sqlci or trafci, then verify that the all the processes are up and running.

swstatus should show at 6 java servers and 2 mysql processes.
sqcheck should indicate all processes are running.

If processes are not running as expected, then:

sqstop to shut down Traodion. If some Trafodion processes do not terminate cleanly, then run ckillall.
swstopall to shut down the Hadoop ecosystem.
swstartall to restart the Hadoop ecosystem.
sqstart to restart Trafodion.

If problems persist please review logs:

$MY_SQROOT/sql/local_hadoop/*/log: Hadoop, HBase, and Hive logs.
$MY_SQROOT/logs Trafodion logs.

You are done and ready to go!

Space shortcuts

Page tree

Versions Compared

Old Version 56

New Version Current

Key

Table of Contents

Supported Platforms

Build prerequisites

Install Required Build Tools

Verify System Limits

Build Trafodion

Build Trafodion

Setup Test Environment

Hadoop is Already Installed

Install a Local Hadoop Environment

Local Hadoop prerequisites

Setup Passwordless SSH

Prepare the Test Environment

If Hadoop is Already Installed

Verify System Limits

Run install_local_hadoop

Usage

Sample Procedure

Sample Procedure

Manage Hadoop Environment

Run Trafodion

Each Time New Source is Downloaded

Run Trafodion

Setup required each time source is downloaded

Start up Trafodion

Create Trafodion metadata

Validate

Start Trafodion

Management Scripts

Create Trafodion Metadata

Validate Your Installation

Troubleshooting Notes