Page History

This document describes the steps required to build ApacheTrafodion software.

<TDB: change hardcoded version number 1.2.0 to be configurable>

Table of Contents

Supported Platforms

• Red Hat or Centos 6.x (6.4 or later) versions are supported as development and production platforms.

...

1. Java version is compatible, it must be 1.7.x. To check your version, do "java -version". Ensure JAVA_HOME environment variable exists and set to your JDK installation.

2. Apache Maven version 3.0.5 or greater is installed and part of your PATH, to check your version do: "mvn --version"
3. Verify that Trafodion source is downloaded or cloned from github
4. Download, build and install additional development tools via  Additional Build Tools.

Check your system limits.  Some of them may need to be increased or Trafodion will not start.
The following are the recommended values:

 ulimit –a

•  core file size                    (blocks, -c) 1000000
• data seg size                    (kbytes, -d) unlimited
• scheduling priority            (-e) 0
• file size                             (blocks, -f) unlimited
• pending signals                (-i) 515196
• max locked memory         (kbytes, -l) 49595556
• max memory size             (kbytes, -m) unlimited
• open files                          (-n) 32000
• pipe size                           (512 bytes, -p) 8
• POSIX message queues  (bytes, -q) 819200
• real-time priority               (-r) 0
• stack size                         (kbytes, -s) 10240
• cpu time                           (seconds, -t) unlimited
• max user processes         (-u) 267263
• virtual memory                 (kbytes, -v) unlimited
• file locks                           (-x) unlimited

Build Steps

Set up Trafodion configuration file

...

Additional development tools are required before building Trafodion  as described Additional Build Tools.  A convenience script script exists that downloads, installs, and builds all these tools in a common directory.  If this convenience script is not used or if any of these additional build tools are not found in the expected location, then the Trafodion configuration file needs to be updated. The Trafodion configuration file template is located in <traf download <download directory>/apache-trafodion-1.2.0-incubating/core/sqf/LocalSettingsTemplate.sh.  To change values, copy this file to your home directory and change its name to .trafodion.  Edit the .trafodion file and update according to the instructions.  Be sure to change the location of your TOOLSDIR to your <tools installation directory>.

   cp <traf<download download directory>/apache-trafodion-1.2.0-incubating/core/sqf/LocalSettingsTemplate.sh ~/.trafodion

 For now, don't change locations for HADOOP_PREFIX, HBASE_HOME, and HIVE_HOME.  There is a later step that describes this process - is this still needed with latest build changes?

Build Trafodion

...

It is recommended that you add TOOLSDIR to your startup shell, for example "export TOOLSDIR=/home/tools"

Build Trafodion

Start a new ssh session

    cd <download directory>/apache-trafodion-1.2.0-incubating
    export $TOOLSDIR=<tools installation directory>
    source ./env.sh

...

Verify build by executing the following statement, this request should return over 90 components:

    sqvers

...

 -u

Prepare the test environment

It is recommended that you test Trafodion by setting up a local version of Hadoop and installing Trafodion on top.  This is enabled by running the 'install_local_hadoop' script.

Use install_local_hadoop

This section describes the steps to use the Trafodion installation script called 'install_local_hadoop' that encapsulates building the product and starting the Hadoop eco-system.  This script uses a Cloudera distribution.downloads compatible versions of Hadoop, HBase, Hive, and MySql and starts Trafodion.

Start a new ssh session and set environment

     make Make sure you have set up password less authentication. You should be able to "ssh localhost" without having to enter a password

Install Hadoop eco-system

Start a new ssh session and set environment

...

     cd <download directory>/apache-trafodion-1.2.0-incubating

...

     source ./env.sh

...

Install Hadoop, HBase, Hive, MySql, and Trafodion

     cd $MY_SQROOT/sql/scripts
     install_local_hadoop
     source ./install_traf_components 

Verify that the install Verify that build completed by running the following command - should report: 6 java servers and 2 mysqld processes are running

...

When this script completes, Hadoop, HBase, Hive, and MySql (used as Hive's metadata repository) have been installed and are started. 

The 'install_local_hadoop' script also creates several helper scripts starting with "sw" in the $MY_SQROOT/sql/scripts directory including:

    swhadoop                               swhbase                             swhdfs
    swhive                                    swjdbc                                 swmysql
    swmysqladmin                        swstartall                             swstarthadoop
    swstarthbase                          swstartmysql                       swstatus
    swstopall                                 swstophadoop                    swstophbase
    swstopmysql                           swuninstall_local_hadoop   swurls.html
    swyarn

To start/stop/check Hadoop environment using Trafodion supplied scripts, you can execute  ‘swstartall’ ,  ‘swstopall’ and ‘swstatus’, and if you need to remove the installation, execute the 'swuninstall_local_hadoop'. 

Use pre-installed Hadoop

<TBD - section to be added>

Run Trafodion

This section describes how to start Trafodion and run operations. 

...

One-time setup

Start a new ssh session and set up environment:

      cd <download directory>/apache-trafodion-1.2.0-incubating
      source ./env.sh

Build using one of the following options: 

    make all          (Build Trafodion, DCS, REST)    OR
    make package       (Build Trafodion, DCS, REST, Client drivers) OR
    make package-all  (Build Trafodion, DCS, REST, Client drivers and Tests for all components

Verify build by executing the following statement, this script should return over 90 components

    sqvers

Install Trafodion components

    install_traf_components 

Test Trafodion build

If you built Trafodion using Build using standard Apache installation, follow the instructions in Test using standard Apache installation.  If you built Trafodion using the Build using local Hadoop , follow the instructions in Test using local Hadoop.

Test using standard Apache installation

<TBD - add instructions on how to .tar up the build. This section needs some work>

Note: All tar files are created in ‘distribution’ folder located at the very top level (incubator-trafodion) during the make step. 

To install a custom Trafodion component you can set various environment variable to overwrite the default tar files found in distribution folder.

Environment variables supported by install_traf_component script are :

DCS_TAR         —  Specify the fully qualified path of  DCS tar file

REST_TAR       —  Specify the fully qualified path of  REST tar file

PHX_TAR         —  Specify the fully qualified path of  Phoenix test tar file

CLIENT_TAR    —  Specify the fully qualified path of  Trafodion client tar file

DCSTEST_TAR —  Specify the fully qualified path of  DCS tests tar file

Follow installation instructions found on the Apache Trafodion wiki site - (Installation ) but modify the step that request you to download tar files from the download directory to use your generated the tar files.

Test using local Hadoop

This section describes how to start up Trafodion and run operations.  It assumes that you have previously run the 'install_local_hadoop' script.

The 'install_local_hadoop' script configures everything needed to run Trafodion.  It also starts Hadoop, Hive, MySql, and HBase.

 Before starting, please check your ulimit.  Some of them may need to be increased.   If incorrect values are used, Trafodion may not start.

The following are the recommended values.

 ulimit –a

•  core file size                    (blocks, -c) 1000000
• data seg size                    (kbytes, -d) unlimited
• scheduling priority            (-e) 0
• file size                             (blocks, -f) unlimited
• pending signals                (-i) 515196
• max locked memory         (kbytes, -l) 49595556
• max memory size             (kbytes, -m) unlimited
• open files                          (-n) 32000
• pipe size                           (512 bytes, -p) 8
• POSIX message queues  (bytes, -q) 819200
• real-time priority               (-r) 0
• stack size                         (kbytes, -s) 10240
• cpu time                           (seconds, -t) unlimited
• max user processes         (-u) 267263
• virtual memory                 (kbytes, -v) unlimited
• file locks                           (-x) unlimited

Start Trafodion

Start a new ssh session and set environment, then perform a one-time step to set up run environment:

    cd $MY_SQROOT/etc
    # delete ms.env, if it exists
    rm ms.env
    cd $MY_SQROOT/sql/scripts
    sqgen 

Start up Trafodion

    cd <download directory>/apache-trafodion-1.2.0-incubating
    source ./env.sh
    cd $MY_SQROOT/sql/scripts
    sqgen 

Important: After sqgen, exit all shells used for Trafodion, and source the env.sh file in a new shell.

...

    swstatus

...

sqstart
    swstophbase
    swstarthbase

Finally, start the Trafodion processes:

    sqstartsqcheck 

Note: In case of any issues and if there is a need to stop and restart a specific Trafodion component, you can use the component based  start/stop scripts.

Checking the status of Trafodion and its components

There are several health check scripts that are available which will provide the status of Trafodion. They are :

sqcheck   (For all of Trafodion)
dcscheck  (For Database Connectivity Service)
rmscheck  (For RMS Server)

...

Create Trafodion metadata

Trafodion is up and running, you can now start up a SQL command line  interface and initialize Trafodion

Start a new ssh session and , set environment and run the sqlci tool:

...

    initialize trafodion;
    exit

Validate 

Test your setup by using "sqlci" or "trafci" (uses DCS to connect to the SQL engine):

...

You are done and ready to go! <TBD: add reference to documentation>

Trafodion test packages

There are several helper scripts provided to run the tests for Trafodion components in your workspace. These scripts are generated based on the tar files that are made available during execution of install_traf_component script

swphoenix {t4 | t2 } — This script will run the Phoenix test using JDBC Type 4 driver or JDBC Type 2 driver

swjdbc —  This script will run JDBC Type 4 tests

...