This document describes the steps required to build ApacheTrafodion software.
Supported Platforms
Red Hat 6.4 or Centos 6.4 versions are supported as development and production platforms.
Required Software
- Install Cloudera or Horton works Hadoop distribution. In situations where you do not have the hadoop distributions already available you can use the install local hadoop script as described below. See section "Set up Hadoop distribution "
Java 1.7.x or greater must be installed. Ensure JAVA_HOME environment environment variable exists and set to your JDK installation.
- Download, build and install additional development tools via Additional Build Tools
- Install the following packages via yum install <package>
alsa-lib-devel | ant | ant-nodeps |
boost-devel | device-mapper-multipath | dhcp |
gcc-c++ | gd | glibc-devel.i686 |
graphviz-perl | gzip | java-1.7.0-openjdk-devel |
libaio-devel | libibcm.i686 | |
libibumad-devel | libibumad-devel.i686 | libiodbc |
libiodbc-devel | librdmacm-devel | librdmacm-devel.i686 |
log4cxx | log4cxx-devel | lua-devel |
lzo-minilzo | net-snmp-devel | net-snmp-perl |
openldap-clients | openldap-devel.i686 | openmotif |
openssl-devel.i686 | openssl-static | perl-Config-IniFiles |
perl-DBD-SQLite | perl-Config-Tiny | perl-Expect |
perl-IO-Tty | perl-Math-Calc-Units | perl-Params-Validate |
perl-Parse-RecDescent | perl-TermReadKey | perl-Time-HiRes |
protobuf-compiler | protobuf-devel | python-qpid |
python-qpid-qmf | qpid-cpp-client | qpid-cpp-client-devel |
qpid-cpp-client-ssl | qpid-cpp-server | qpid-cpp-server-ssl |
qpid-qmf | qpid-tools | readline-devel |
saslwrapper | sqlite-devel | |
libXext-devel | libX11-devel | libXau-devel |
unixODBC | unixODBC-devel | uuid-perl |
xinetd | xerces-c-devel |
Set up Hadoop distribution (Install Hadoop, Hbase, Hive to local workspace)
You can use single node standalone Apache install OR use Trafodion supplied ‘install_local_hadoop’ script pre-configured to install Cloudera distribution for a single node. When using Trafodion supplied Hadoop install script you can do the following:
Make sure you have set up password less authentication. Basically you should be able to "ssh localhost" without having to enter a password
- Download latest Apache Trafodion source from Apache Incubator https://github.com/apache/incubator-trafodion
- Using ssh, set Trafodion environment
cd incubator-trafodion; . ./env.sh
cd $MY_SQROOT/sql/scripts
Execute the script ‘install_local_hadoop’
Note:
This script will download Hadoop and HBase jar files from the internet. To avoid this overhead for future executions of the script, you can save the downloaded files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to that directory. The files to save are: $MY_SQROOT/sql/local_hadoop/*.tar.gz $MY_SQROOT/sql/local_hadoop/tpcds/tpcds_kit.zip.
‘install_local_hadoop –p rand' — will start with any random port number between 9000 and 49000
OR
‘install_local_hadoop –p < specify a port # >' — will start with port number specified
OR
‘install_local_hadoop' — will use default port numbers for all services
To start/stop/check Hadoop environment when using Trafodion supplied Hadoop install script, you can execute ‘swstartall’ , ‘swstopall’ and ‘swstatus’
For Hadoop installs that did not use Trafodion supplied Hadoop install script, please update HBase configuration as shown below and restart HBase
For hbase-site.xml:
<property>
<name>hbase.client.scanner.caching</name>
<value>100</value>
</property>
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>60000</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>
org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver,
org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint,
org.apache.hadoop.hbase.coprocessor.AggregateImplementation
</value>
</property>
<property>
<name>hbase.hregion.impl</name>
<value>org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion</value>
</property>
For hbase-env.xml:
export HBASE_CLASSPATH=${HBASE_TRXDIR}/${HBASE_TRX_JAR}
To compile and configure Trafodion and its components
Set your TOOLSDIR environment variable to the location of the components installed via Additional Build Tools
If you have not already downloaded Apache Trafodion source now you can download it from here https://github.com/apache/incubator-trafodion
Using a new ssh session, set Trafodion environment
cd incubator-trafodion
. ./env.sh
make all (Build Trafodion, DCS, REST) OR
make package (Build Trafodion, DCS, REST, Client drivers) OR
make package-all (Build Trafodion, DCS, REST, Client drivers and Tests for all components)
b. . cd $MY_SQROOT/sql/scripts
c. Execute script , ‘install_traf_components’. Based on the tar files available in the distribution folder this script will install the Trafodion components.
Note: All tar files will be created in ‘distribution’ folder located at the very top level (incubator-trafodion)
To install a custom Trafodion component you can set various environment variable to overwrite the default tar files found in distribution folder.
Environment variables supported by install_traf_component script are :
DCS_TAR — Specify the fully qualified path of DCS tar file
REST_TAR — Specify the fully qualified path of REST tar file
PHX_TAR — Specify the fully qualified path of Phoenix test tar file
CLIENT_TAR — Specify the fully qualified path of Trafodion client tar file
DCSTEST_TAR — Specify the fully qualified path of DCS tests tar file
Starting Trafodion and its components
Using a new ssh session,
cd incubator-trafodion; Execute . ./env.sh
cd $MY_SQROOT/sql/scripts
Execute the script, ‘sqgen’ and then start using the script ‘sqstart’
Note: In case of any issues and if there is a need to stop and restart a specific Trafodion component, you can use the component based start/stop scripts.
Component | Start script | Stop script |
For all of Trafodion | sqstart | sqstop |
For DCS (Database Connectivity Service) | dcsstart | dcsstop |
For REST server | reststart | reststop |
For LOB server | lobstart | lobstop |
For RMS server | rmsstart | rmsstop |
|
|
|
Checking the status of Trafodion and its components
There are several health check scripts that are available which will provide the status of Trafodion. They are :
sqcheck (For all of Trafodion)
dcscheck (For Database Connectivity Service)
rmscheck (For RMS Server)
Creating Trafodion metadata
Using a new ssh session,
cd incubator-trafodion; . ./env.sh
cd $MY_SQROOT/sql/scripts
Use sqlci (direct to sql engine) or trafci (uses DCS to connect to SQL engine)
Execute the sql initialization script via sqlci or trafci, ‘initialize trafodion’
Testing Trafodion
There are several helper scripts provided to run the tests for Trafodion components in your workspace. These scripts are generated based on the tar files that are made available during execution of install_traf_component script
swphoenix {t4 | t2 } — This script will run the Phoenix test using JDBC Type 4 driver or JDBC Type 2 driver
swjdbc — This script will run JDBC Type 4 tests
swpyodbc — This script will install Linux drivers run ODBC tests using Linux driver