Versions Compared

Old Version 31

changes.mady.by.user Radar Lei

Saved on

compared with

New Version 32

changes.mady.by.user Caleb Welton

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updated formatting of dependency section to be tabs for the different options

Table of Contents

How to Build

Option 1 (with pre-built docker image)

...

Setup an environment with the dependencies installed

Panel
Tabs Container
idbuild-tabs
directionhorizontal
Tabs Page
iddocker-tab
titleDocker

Build with Prebuilt Docker Image

 

Probably the simplest way to get started with the build is starting with the community developed docker image with all the project dependencies pre-installed.

 

To use the docker image follow the steps on:

...

https://hub.docker.com/r/mayjojo/hawq-devel/

...

Tabs Page
idtab-yum
titleYum Install

Install

...

dependencies on CentOS7.x

...

OS requirement

Linux ( tested on centos 7.x).

Dependencies

(CentOS7 user can follow easy steps provided by Zhanwei Wang)

 

Code Block
languagebash
curl -L "https://bintray.com/wangzw/rpm/rpm" -o /etc/yum.repos.d/bintray-wangzw-rpm.repo

...


yum -y swap -- remove systemd-container systemd-container-libs -- install systemd systemd-libs && \

...


yum install -y epel-release && \

...


yum makecache && \

...


yum install -y man passwd sudo tar which git mlocate links make bzip2 net-tools \

...


autoconf automake libtool m4 gcc gcc-c++ gdb bison flex cmake gperf maven indent \

...


libuuid-devel krb5-devel libgsasl-devel expat-devel libxml2-devel \

...


perl-ExtUtils-Embed pam-devel python-devel libcurl-devel snappy-devel \

...


thrift-devel libyaml-devel libevent-devel bzip2-devel openssl-devel \

...


openldap-devel protobuf-devel readline-devel net-snmp-devel apr-devel \

...


libesmtp-devel xerces-c-devel python-pip json-c-devel libhdfs3-devel \

...


apache-ivy java-1.7.0-openjdk-devel \

...


openssh-clients openssh-server && \

...


yum clean all

...


yum makecache && yum install -y postgresql-devel && \

...


pip --retries=50 --timeout=300 install pg8000 simplejson unittest2 pycrypto pygresql pyyaml lockfile paramiko psi && \

...


pip --retries=50 --timeout=300

...

 install http://darcs.idyll.org/~t/projects/figleaf-0.6.1.tar.

...

gz && \

...


pip --retries=50 --timeout=300

...

 install http://sourceforge.net/projects/pychecker/files/pychecker/0.8.19/pychecker-0.8.19.tar.gz/

...

download && \

...


yum erase -y postgresql postgresql-libs postgresql-devel && \

...


yum clean all

 

...

Tabs Page
titleCompile dependencies
classtabs-compile-dependencies

Build dependencies yourself

OS requirement

Linux ( tested on redhat 6.x).

Dependencies

There are several dependencies (see the following table) you must install before building HAWQ. To build Apache HAWQ, gcc and some dependencies are needed. The libraries are tested on the given versions. Most of the dependencies can be installed through yum. Other dependencies should be installed through the source tarball. Typically you can use "./configure && make && make install" to install from source tarball.

Libraries that must be installed using source tarball.


NameVersionDownload URL
json-c
0.9

http://oss.metaparadigm.com/json-c/json-c-0.9.tar.gz

boost
1.56
http://sourceforge.net/projects/boost/files/boost/1.56.0/boost_1_56_0.tar.bz2
thrift
0.9.1-1
http://archive.apache.org/dist/thrift/0.9.1/thrift-0.9.1.tar.gz
protobuf
2.5.0
https://github.com/google/protobuf/tree/v2.5.0
curl
7.44.0
http://www.curl.haxx.se/download/curl-7.44.0.tar.gz
libhdfs3
latest version
https://github.com/PivotalRD/libhdfs3.git
libyarn
 
Code shipped with Apache HAWQ (under incubator-hawq/depends/)
  • cd depends/libyarn/
  • mkdir build
  • cd build
  • ../bootstrap --prefix=/usr/local/
  • make
  • sudo make install

 

 

Libraries that can be installed through yum.

NameVersoin
epel-release6-8
make
3.81
gcc
4.4.7
gcc-c++
4.4.7
gperf
3.0.4
snappy-devel
1.1.3
bzip2-devel 
1.0.6
python-devel 
2.6.2
libevent-devel
1.4.6
krb5-devel
1.11.3
libuuid-devel
2.26.2
libgsasl-devel
1.8.0
libxml2-devel 
2.7.8
zlib-devel
1.2.3
readline-devel
6
openssl-devel
0.9.8
bison-devel
2.5
apr-devel
1.2.12
libyaml-devel
0.1.1
flex-devel
2.5.35

Get the code and Compile

Once you have an environment with the necessary dependencies installed, the next step is to get the code and build HAWQ

  • The Apaceh Apache HAWQ source code can be obtained from the GitHub: https://github.com/apache/incubator-hawq. 
  • Get source code
  • The code directory is CODEHOME/incubator-hawq. Then cd CODEHOME/incubator-hawq and build Apache HAWQ under this directory.
  • Run command to generate makefile.
    • ./configure
  • Or you could use --prefix=/hawq/install/path to change the Apache HAWQ install path.
    • ./configure --prefix=/hawq/install/path
  • You can also run the command with --help for more configuration.
    • ./configure --help
  • Run command to build
    • make
  • To build concurrently , run make with -j option.
    • make -j8

Install HAWQ

  • To install Apache HAWQ, run command
    • make install

Test In HAWQ

  • Unit test. To do unit test, go to the src/backend and run unittest.
    • cd src/backend
    • make unittest-check
  • Installcheck-good test. After installing HAWQ, please ensure HDFS work before initializing HAWQ. 
    • source /install/dir/greenplum_path.sh
    • hawq init cluster
    • make installcheck-good

Install YARN (Optional)

If you want to integrate with YARN for resource management, you need to install YARN first.

...