Versions Compared

Old Version 22

changes.mady.by.user Zhanwei Wang

Saved on

compared with

New Version 23

changes.mady.by.user Xiang Sheng

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

How to Build

Option 1 (with pre-built docker image)

Follow the steps on: https://hub.docker.com/r/mayjojo/hawq-devel/

Option 2 (build the dependencies by yourself)

OS requirement

Currently, only Redhat 6.5 and CentOS 6.5 are verified to work. 

  • install redhat 6.5 basic server

  • choose install development tools 

Linux ( tested on redhat 6.x).

Dependencies

There are several dependencies (see the following table) you must install before building HAWQ. The steps in "Compile and Install" column typically install the headers to /usr/local/include and libraries to /usr/local/lib or /usr/local/lib64. If you want to install the lib in another place, you can use the --prefix=/the/path option when you run configure. For example: ./configure --prefix=/the/path.  To build Apache HAWQ, gcc and some dependencies are needed. The libraries are tested on the version given. Most of the dependencies could be installed through yum. Other dependencies should be installed through the source tarball, which means the typically make 3 steps "./configure && make && make install".

Libraries should be installed using source tarball.

NameVersionDownload URLCompile and Install
GCC
4.4.7

https

gperf3.0.4http

://

ftp

gcc.gnu.org

/gnu/gperf/gperf-3.0.4.tar.gz

/

configure

GNU make

sudo make install

snappy1.1.3https://github.com/google/snappy/releases/download/1.1.3/snappy-1.1.3.tar.gz

./configure

make

sudo make install
     
3.81
bzip21.0.6

http://

www

ftp.

bzip

gnu.org/

1.0.6/bzip2-1.0.6.tar.gz

make

sudo make install
python2.6.2https://www.python.org/ftp/python/2.6.2/Python-2.6.2.tgz

./configure

make

sudo make install
libevent1.4.6https://github.com/downloads/libevent/libevent/libevent-1.4.6-stable.tar.gz

./configure

make

sudo make install

gnu/make/

json-c
0.9

http://oss.metaparadigm.com/json-c/json-c-0.9.tar.gz

./configure

make

sudo make install		krb5
boost
1.56
1.11.3
http://websourceforge.mit.edunet/kerberosprojects/distboost/krb5files/1.11boost/krb5-1.11.3-signed.tar
./configure
make
sudo make install
libuuid2.26.2http://mirrors.163.com/centos/6.7/os/x86_64/Packages/libuuid-devel-2.17.2-12.18.el6.x86_64.rpmrpm -i libuuid-devel-2.17.2-12.18.el6.x86_64.rpm
libgsasl1.8.0ftp://ftp.gnu.org/gnu/gsasl/libgsasl-1.8.0.tar.gz
./configure
make
sudo make install
libxml22.7.8http://xmlsoft.org/sources/libxml2-2.7.8.tar.gz
./configure
make
sudo make install
cd /usr/local/include/
sudo ln -s libxml2/libxml libxml
56.0/boost_1_56_0.tar.bz2
thrift
0.9.1-1
zlib1.2.3
http://zlib.net/fossils/zlib-1.2.3.tar.gz
./configure --prefix=/usr
make
sudo make install
readline6.0ftp://ftp.gnu.org/gnu/readline/readline-6.0.tar.gz
./configure
make
sudo make install
 (you may need to run ldconfig)
openssl0.9.8ftp://ftp.openssl.org/source/opensslarchive.apache.org/dist/thrift/0.9.1/thrift-0.9.8ze1.tar.gz
./config --prefix=/usr/local --openssldir=/usr/local/openssl
makesudo make install
protobuf
2.5.0
https://github.com/google/protobuf/tree/v2.5.0
curl
./configure
make
sudo make install		boost1.56
7.44.0
http://sourceforge.net/projects/boost/files/boost/1.56.0/boost_1_56_0.tar.bz2
./bootstrap.sh
./b2
sudo ./b2 install
bison2.5http://ftp.gnu.org/gnu/bison/bison-2.5.tar.gz
./configure
make
sudo make install
thrift0.9.1-1http://archive.apache.org/dist/thrift/0.9.1/thrift-0.9.1.tar.gz
./configure
make
sudo make install
apr1.2.12http://archive.apache.org/dist/apr/apr-1.2.12.tar.gz
./configure
make
sudo make install
libyaml0.1.1
http://pyyaml.org/download/libyaml/yaml-0.1.1.tar.gz
./configure
make
sudo make install
curl7.44.0http://www.curl.haxx.se/download/curl-7.44.0.tar.gz
./configure
make
sudo make install
flex2.5.35http://sourceforge.net/projects/flex/files/flex-2.5.35.tar.gz/download
./configure
make
sudo make install
www.curl.haxx.se/download/curl-7.44.0.tar.gz
libhdfs3
latest version
https://github.com/PivotalRD/libhdfs3.git
libyarn
 
Code shipped with Apache HAWQ (under incubator-hawq/depends/)
 
 
Libraries could be installed through yum.
Get code : git clone
Install dependencies follow README.md
Run:
   mkdir build
   cd build 
   ../bootstrap --prefix=/usr/local/
   make
   sudo make install
NameVersoin
gperf
3.0.4
snappy-devel
1.1.3
bzip2-devel 
1.0.6
python-devel 
2.6.2
libevent-devel
1.4.6
krb5-devel
1.11.3
libuuid-devel
2.26.2
libgsasl-devel
1.8.0
libxml2-devel 
2.7.8
zlib-devel
1.2.3
readline-devel
6
openssl-devel
0.9.8
bison-devel
2.5
apr-devel
1.2.12
libyaml-devel
0.1.1
flex-devel
2.5.35
libhdfs3latest versionhttps://github.com/PivotalRD/libhdfs3.git
Get code and build HAWQ
  • The 
...
  • Apaceh HAWQ 
...
  • source code can be obtained from the 
...
  • GitHub: https://github.com/apache/incubator-hawq. 
  • Get source code
      # git 
    • The code directory is CODEHOME/incubator-hawq. Then cd CODEHOME/incubator-hawq and build Apache HAWQ under this directory.
    • Run command to generate makefile.
      • ./configure
    • Or you could user --prefix=/hawq/install/path to change the Apache HAWQ install path.
      • ./configure 
      Install libyarn
      # cd depends/libyarn/
      # mkdir build
      # cd build
      # ../bootstrap 
      •  --prefix=/
      usr
      • hawq/
      local
      • install/

      # make
      # sudo make install
    • Go to top hawq source directory
      # ./configure
      # make -j8
      # sudo make install
    Install HDFS
    ...
      • path
    • You also could run the command with --help for more configuration.
      • ./configure --help
    • Run command to build
      • make
    • To build concurrently , run make with -j option.
      • make -j8
    Install HAWQ
    • To install Apache HAWQ, run command
      • make install
    Test In HAWQ
    • Unit test. To do unit test, go to the src/backend and run unittest check.
      • cd src/backend
      • make unittest-check
    • Install check. After installed Apache HAWQ , ensure HDFS work before init Apache HAWQ.Then init Apache HAWQ.To do install check, run the command below under incubator-hawq.
      • source /install/dir/greenplum_path.sh
      • hawq init cluster
      • make installcheck-good
    Install YARN (Optional)
    If you want to use YARN integration feature, you need to install YARN first.
    Init and start HAWQ
    ...
    • Install libyarn
      • cd depends/libyarn/
      • mkdir build
      • cd build
      • ../bootstrap --prefix=/usr/local/
      • make
      • sudo make install
    Init and Start Apache HAWQ
    • Before initializing HAWQ, you need to install HDFS and make sure it works
      • source /install/dir/greenplum_path.sh
    ...
      • hawq init cluster
    ...
      • hawq start cluster
    Connect and 
    ...
    Run basic queries
    ...
    • psql -d postgres
    ...
    • create table t ( i int );
    ...
    • insert into t values(1);
    ...
    • select * from t;