Build dependencies yourself ( tested on Red Hat 6.X).Dependencies There are several dependencies (see the following table) you must install before building HAWQ. To build Apache HAWQ, gcc and some dependencies are needed. The libraries are tested on the given versions. Most of the dependencies can be installed through yum. Other dependencies should be installed through the source tarball. Typically you can use "./configure && make && make install" to install from source tarball. Libraries that must be installed using source tarball.
You might need to run "ldconfig -p <LIBRARY_INSTALL_PATH>" after installing them. For thrift build, you might need "--without-tests" for configure. Install maven:
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
sudo yum install -y apache-maven Install pip: wget https://bootstrap.pypa.io/get-pip.py python get-pip.py pip --retries=50 --timeout=300 install paramiko pycrypto Libraries that can be installed through yum. | Name | Versoin |
|---|
| epel-release | 6-8 | | make | 3.81
| | gcc | >=4.7.2 | | gcc-c++ | >=4.7.2
| gperf
| 3.0.4
| snappy-devel
| 1.1.3
| bzip2-devel
| 1.0.6
| python-devel
| 2.6.2
| libevent-devel
| 1.4.6
| krb5-devel
| 1.11.3
| libuuid-devel
| 2.26.2
| libgsasl-devel
| 1.8.0
| libxml2-devel
| 2.7.8
| zlib-devel
| 1.2.3
| readline-devel
| 6
| openssl-devel
| 0.9.8
| bison
| 1.875 | apr-devel
| 1.2.12
| libyaml-devel
| 0.1.1
| flex
| >2.5.4 | | lcov | 1.12 | | libesmtp-devel | |
Default version of gcc in Red Hat/CentOS 6.X is 4.4.7 or lower, you can quickly upgrade gcc following instructions below: | Code Block |
|---|
| cd /etc/yum.repos.d
# make sure you have root permission
wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo
# install higher version using devtoolset-2
yum install devtoolset-2-gcc devtoolset-2-binutils devtoolset-2-gcc-c++
# Start using software collections
scl enable devtoolset-2 bash |
You will need to install python packages same as those which are required for Red Hat/CentOS 7. OS requirement kernel.shmmax = 1000000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 0
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 200000
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1281 65535
net.core.netdev_max_backlog = 200000
vm.overcommit_memory = 2
fs.nr_open = 3000000
kernel.threads-max = 798720
kernel.pid_max = 798720
# increase network
net.core.rmem_max=2097152
net.core.wmem_max=2097152- Execute the following command to apply your updated /etc/sysctl.conf file to the operating system configuration:
sysctl -p
- Use a text editor to edit the /etc/security/limits.conf file. Add the following definitions in the exact order that they are listed
* soft nofile 2900000
* hard nofile 2900000
* soft nproc 131072
* hard nproc 131072 |