...
Red Hat or Centos 6.x (6.4 or later) versions are supported as development and production platforms.
...
Build prerequisites
You need to install the following packages before you can install Apache Trafodion.
...
Note |
---|
You should export JAVA_HOME in your .bashrc or .profile file. |
Setup Passwordless SSH
Check to see if you have passwordless SSH setup.
Code Block | ||||
---|---|---|---|---|
| ||||
ssh localhost
Last login: Fri Nov 6 22:44:00 2015 from 192.168.1.9 |
If passwordless SSH is not setup, please do so now. The following is an example of setting up passwordless SSH using id_rsa keys. You can choose the method that best represents your environment.
If you already have an existing set of ssh keys. Simply copy both the id_rsa.pub and id_rsa to your ~/.ssh directory.
Then, do the following:
Code Block | ||||
---|---|---|---|---|
| ||||
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 .. |
If you need to create your keys first, then do the following:
Code Block | ||||
---|---|---|---|---|
| ||||
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa.pub
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 .. |
Verify Trafodion Download
Verify that the Trafodion source has been either:
- Downloaded and unpackaged.
- Cloned from github.
If not, please do so now.
To download from github, refer to Contributor Workflow - Code/Docs.
Otherwise downloaded and untar the source tar file from Apache Trafodion Incubator release.
Install Required Build Tools
Trafodion requires a set of tools to be installed in order to build. Refer to Required Build Tools for instructions. One of the tools that is built if it does not already exist is Maven. At this time, you should verify that Maven is part of your path. To verify that Maven is available do:
mvn --version.
If not found, then added it to your path. For ease of use, you should add it to .bashrc or .profile. Example:
PATH=$PATH:<tool installation directory>/apache-maven-3.3.3/bin
Verify System Limits
Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.
Code Block | ||||
---|---|---|---|---|
| ||||
ulimit –a
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 515196
max locked memory (kbytes, -l) 49595556
max memory size (kbytes, -m) unlimited
open files (-n) 32000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 267263
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
|
Please refer to this article for information on how you change system limits.
Build Trafodion
Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.
The <Trafodion source directory> is source tree base for Trafodion.
The <tools installation directory> is where Trafodion required tools are located.
This will build a debug version of Trafodion.
Code Block | ||
---|---|---|
| ||
cd <Trafodion source directory>
export TOOLSDIR=<tools installation directory>
source ./env.sh
|
Build using one of the following options:
Command | What It Builds |
---|---|
make all | Trafodion, DCS, and REST. |
make package | Trafodion, DCS, REST, and Client Drivers. |
make package-all | Trafodion, DCS, REST, Client Drivers, and tests for all components. |
If the build fails, you might want to rerun the make step. Trafodion downloads many dependencies and sometimes one of the download operations fail. Rerunning the build generally works.
Verify the build:
Verify Trafodion Download
Verify that the Trafodion source has been either:
- Downloaded and unpackaged.
- Cloned from github.
If not, please do so now.
To download from github, refer to Contributor Workflow - Code/Docs.
Otherwise downloaded and untar the source tar file from Apache Trafodion Incubator release.
Install Required Build Tools
Trafodion requires a set of tools to be installed in order to build. Refer to Required Build Tools for instructions. One of the tools that is built if it does not already exist is Maven. At this time, you should verify that Maven is part of your path. To verify that Maven is available do:
mvn --version.
If not found, then added it to your path. For ease of use, you should add it to .bashrc or .profile. Example:
PATH=$PATH:<tool installation directory>/apache-maven-3.3.3/bin
Build Trafodion
Start a new ssh session. Use the following commands to set up the Trafodion environmental variables.
The <Trafodion source directory> is source tree base for Trafodion.
The <tools installation directory> is where Trafodion required tools are located.
This will build a debug version of Trafodion.
Code Block | ||
---|---|---|
| ||
cd <Trafodion source directory>
export TOOLSDIR=<tools installation directory>
source ./env.sh
|
Build using one of the following options:
Command | What It Builds |
---|---|
make all | Trafodion, DCS, and REST. |
make package | Trafodion, DCS, REST, and Client Drivers. |
make package-all | Trafodion, DCS, REST, Client Drivers, and tests for all components. |
If the build fails, you might want to rerun the make step. Trafodion downloads many dependencies and sometimes one of the download operations fail. Rerunning the build generally works.
Verify the build:
Code Block | ||||
---|---|---|---|---|
| ||||
$ sqvers -u
MY_SQROOT=/home/centos/apache-trafodion-1.3.0-incubating/core/sqf
who@host=centos@mysystem
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.1.3.el6.x86_64
redhat=6.7
NO patches
Most common Apache_Trafodion Release 1.3.0 (Build debug [centos], branch -, date 06Nov15)
UTT count is 1
[7] Release 1.3.0 (Build debug [centos], branch -, date 06Nov15)
export/lib/hbase-trx-cdh5_3-1.3.0.jar
export/lib/hbase-trx-hbase_98_4-1.3.0.jar
export/lib/hbase-trx-hdp2_2-1.3.0.jar
export/lib/hbase-trx-mapr4_0-1.3.0.jar
export/lib/sqmanvers.jar
export/lib/trafodion-dtm-1.3.0.jar
export/lib/trafodion-sql-1.3.0.jar |
The output show several jar files. The number of files differs based on the version of Trafodion you downloaded.
Setup Test Environment
You should test your installation using:
- Trafodion installation on a system that already has a compatible version of Hadoop installed
- A local Hadoop environment created by the install_local_hadoop script
Your installation approach depends on whether you already have installed Hadoop.
Hadoop is Already Installed
Build binary tar files and then install Trafodion following instructions described in Installation.
Code Block | ||||
---|---|---|---|---|
| ||||
cd <Trafodion source directory>
make package |
The binary tar files will be created in <Trafodion source directory>/distribution directory.
Install a Local Hadoop Environment
Local Hadoop prerequisites
Setup Passwordless SSH
Check to see if you have passwordless SSH setup.
Code Block | ||||
---|---|---|---|---|
| ||||
ssh localhost
Last login: Fri Nov 6 22:44:00 2015 from 192.168.1.9 |
If passwordless SSH is not setup, please do so now. The following is an example of setting up passwordless SSH using id_rsa keys. You can choose the method that best represents your environment.
If you already have an existing set of ssh keys. Simply copy both the id_rsa.pub and id_rsa to your ~/.ssh directory.
Then, do the following:
Code Block | ||||
---|---|---|---|---|
| ||||
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 .. |
If you need to create your keys first, then do the following:
Code Block | ||||
---|---|---|---|---|
| ||||
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa.pub
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys; cd ~/.ssh; chmod 700 .. |
Verify System Limits
Please check that the system limits in your environment are appropriate for Apache Trafodion. If they are not, then you will need to increase the limits or Trafodion cannot start.
Code Block | ||||
---|---|---|---|---|
| ||||
ulimit –a
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 515196
max locked memory (kbytes, -l) 49595556
max memory size (kbytes, -m) unlimited
open files | ||||
Code Block | ||||
| ||||
$ sqvers -u MY_SQROOT=/home/centos/apache-trafodion-1.3.0-incubating/core/sqf who@host=centos@mysystem JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91.x86_64 SQ_MBTYPE=64d (64-debug) linux=2.6.32-504.1.3.el6.x86_64 redhat=6.7 NO patches Most common Apache_Trafodion Release 1.3.0 (Build debug [centos], branch -, date 06Nov15) UTT count is 1 [7] Release 1.3.0 (Build debug [centos], branch -, date 06Nov15) export/lib/hbase-trx-cdh5_3-1.3.0.jar export/lib/hbase-trx-hbase_98_4-1.3.0.jar export/lib/hbase-trx-hdp2_2-1.3.0.jar export/lib/hbase-trx-mapr4_0-1.3.0.jar export/lib/sqmanvers.jar export/lib/trafodion-dtm-1.3.0.jar export/lib/trafodion-sql-1.3.0.jar |
The output show several jar files. The number of files differs based on the version of Trafodion you downloaded.
Setup Test Environment
You should test your installation using:
- A local Hadoop environment
- Trafodion installation
Your installation approach depends on whether you already have installed Hadoop.
If Hadoop is Already Installed
Build binary tar files and then install Trafodion following instructions described in Installation.
Code Block | ||||
---|---|---|---|---|
| ||||
cd <Trafodion source directory>
make package |
The binary tar files will be created in <Trafodion source directory>/distribution directory.
...
(-n) 32000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 267263
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
|
Please refer to this article for information on how you change system limits.
Run install_local_hadoop
The install_local_hadoop script downloads compatible versions of Hadoop, HBase, Hive, and MySQL. Then, it starts Trafodion.
Tip | ||
---|---|---|
| ||
install_local_hadoop downloads Hadoop, HBase, Hive, and MySql jar files from the internet. To avoid this overhead, you can download the required files into a separate directory and set the environment variable MY_LOCAL_SW_DIST to point to this directory. |
...
Command | What It Does |
---|---|
install_local_hadoop | Uses default ports for all services. |
install_local_hadoop -p fromDisplay | Start Hadoop with a port number range determined from the DISPLAY environment variable. |
install_local_hadoop -p rand | Start with any random port number range between 9000 and 49000. |
install_local_hadoop -p <port > | Start with the specified port number. |
...