THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
How to Build
Setup an environment with the dependencies installed
Install dependencies on MAC
brew install protobuf protobuf-c Gsasl openssl boost thrift json-c ccache snappy libyaml libevent python brew tap brona/iproute2mac brew install iproute2mac brew install postgresql sudo pip --retries=50 --timeout=300 install pygresql (please verify if python command ‘import _pg’ works after installation) sudo pip --retries=50 --timeout=300 install unittest2 pycrypto lockfile paramiko psi (use system default gcc) sudo pip --retries=50 --timeout=300 install http://sourceforge.net/projects/pychecker/files/pychecker/0.8.19/pychecker-0.8.19.tar.gz/download (use system default gcc) brew uninstall postgresql Install libhdfs3: git clone https://github.com/Pivotal-DataFabric/libhdfs3 cd libhdfs3 mkdir build cd build ../bootstrap --prefix=/usr/local/ make -j8 make install
Install dependencies on CentOS7.x
OS requirement
Linux ( tested on centos 7.x).
Dependencies
(CentOS7 user can follow easy steps provided by Zhanwei Wang)
curl -L "https://bintray.com/wangzw/rpm/rpm" -o /etc/yum.repos.d/bintray-wangzw-rpm.repo yum install -y epel-release yum makecache yum install -y man passwd sudo tar which git mlocate links make bzip2 net-tools \ autoconf automake libtool m4 gcc gcc-c++ gdb bison flex cmake gperf maven indent \ libuuid-devel krb5-devel libgsasl-devel expat-devel libxml2-devel \ perl-ExtUtils-Embed pam-devel python-devel libcurl-devel snappy-devel \ thrift-devel libyaml-devel libevent-devel bzip2-devel openssl-devel \ openldap-devel protobuf-devel readline-devel net-snmp-devel apr-devel \ libesmtp-devel xerces-c-devel python-pip json-c-devel libhdfs3-devel \ apache-ivy java-1.7.0-openjdk-devel \ openssh-clients openssh-server yum install -y postgresql-devel pip --retries=50 --timeout=300 install pg8000 simplejson unittest2 pycrypto pygresql pyyaml lockfile paramiko psi pip --retries=50 --timeout=300 install http://darcs.idyll.org/~t/projects/figleaf-0.6.1.tar.gz pip --retries=50 --timeout=300 install http://sourceforge.net/projects/pychecker/files/pychecker/0.8.19/pychecker-0.8.19.tar.gz/download yum erase -y postgresql postgresql-libs postgresql-devel
Build dependencies yourself
OS requirement
Linux ( tested on redhat 6.x).
Dependencies
There are several dependencies (see the following table) you must install before building HAWQ. To build Apache HAWQ, gcc and some dependencies are needed. The libraries are tested on the given versions. Most of the dependencies can be installed through yum. Other dependencies should be installed through the source tarball. Typically you can use "./configure && make && make install" to install from source tarball.
Libraries that must be installed using source tarball.
| Name | Version | Download URL |
|---|---|---|
| 0.9 | |
| | http://sourceforge.net/projects/boost/files/boost/1.56.0/boost_1_56_0.tar.bz2 |
| | http://archive.apache.org/dist/thrift/0.9.1/thrift-0.9.1.tar.gz |
| | https://github.com/google/protobuf/tree/v2.5.0 |
| | http://www.curl.haxx.se/download/curl-7.44.0.tar.gz |
| | https://github.com/PivotalRD/libhdfs3.git |
Libraries that can be installed through yum.
| Name | Versoin |
|---|---|
| epel-release | 6-8 |
| make | |
| gcc | |
| gcc-c++ | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
Build with Prebuilt Docker Image
Probably the simplest way to get started with the build is starting with the community developed docker image with all the project dependencies pre-installed.
To use the docker image follow the steps on: https://hub.docker.com/r/mayjojo/hawq-devel/
Get the code and Compile
Once you have an environment with the necessary dependencies installed, the next step is to get the code and build HAWQ
- The Apache HAWQ source code can be obtained from the GitHub: https://github.com/apache/incubator-hawq.
- Get source code
- The code directory is /CODEHOME/incubator-hawq. Then cd /CODEHOME/incubator-hawq and build Apache HAWQ under this directory.
Install libyarn
cd /CODEHOME/incubator-hawq/depends/libyarnmkdir build
cd build
../bootstrap --prefix=/usr/local/
make -j8
make install
- Run command to generate makefile.
- Or you could use --prefix=/hawq/install/path to change the Apache HAWQ install path.
- You can also run the command with --help for more configuration.
- Note: If
./configurecomplains that libyarn is missing, it is provided under ./depends/libyarn. Please follow the README file to install libyarn. You may need to runldconfigafter libyarn is installed.- Run command to build
- To build concurrently , run make with -j option.
make -j8
Install HAWQ
- To install Apache HAWQ, run command
make install
Test In HAWQ
- Unit test. To do unit test, go to the src/backend and run unittest.
cd src/backend
make unittest-check
- Installcheck-good test. After installing HAWQ, please ensure HDFS work before initializing HAWQ.
source /install/dir/greenplum_path.sh
hawq init cluster
make installcheck-good
Install YARN (Optional)
If you want to integrate with YARN for resource management, you need to install YARN first.
Init and Start/Stop Apache HAWQ
- Before initializing HAWQ, you need to install HDFS and make sure it works
- source /install/dir/greenplum_path.sh
- hawq init cluster (after initialization, HAWQ is started by default)
- Now you can stop/restart/start the cluster by using: hawq stop/restart/start cluster
Connect and Run basic queries
- psql -d postgres
- create table t ( i int );
- insert into t values(1);
- select * from t;