tocThis page describes how to build Impala from source and how to configure and run Impala in a single node development environment. 

Prerequisites and Requirements

Hardware Requirements

• Kernel version: ??
• CPU (support for SS3?)
• Minimum Memory: 8GB

• CPU must support at least SSSE3
• Minimum memory: 16GB
• Hard disk space: 120GB (for the test data)

Supported Operating Systems

Linux

• Ubuntu 14.04CentOS 6.5, 16.04, 18.04, 20.04
• CentOS 7
• See bootstrap_development.sh for other supported versions (this wiki page may be stale).

OS X

Not supported

Windows

Not supported

Packages

These instructions are installing the prerequisite packages and configuring for Impala. 

Ubuntu 14.04

# Install Java 7
sudo add-apt-repository ppa:webupd8team/java -y
sudo apt-get update -y
sudo apt-get install oracle-jdk7-installer -y
 
# Install required libraries and packages
sudo apt-get install git build-essential cmake bison flex pkg-config libsasl2-dev autoconf automake libtool maven subversion doxygen libbz2-dev zlib1g-dev python-pip python-setuptools python-dev libssl-dev libboost-all-dev postgresql liblzo2-dev lzop -y
 
# Install Python packages
sudo pip install allpairs pytest pytest-xdist paramiko texttable prettytable sqlparse psutil==0.7.1 pywebhdfs gitpython jenkinsapi boto3

# Configuring Boost
# Newer versions of boost need to be configured to work with Impala. 
# Impala relies on certain boost libraries ending with a "-mt" extension. 
# You need to add symlinks to boost_filesystem, boost_regex, and boost_system.
cd /usr/lib/x86_64-linux-gnu
sudo ln -s libboost_filesystem.so libboost_filesystem-mt.so
sudo ln -s libboost_filesystem.a libboost_filesystem-mt.a
sudo ln -s libboost_system.so libboost_system-mt.so
sudo ln -s libboost_system.a libboost_system-mt.a
sudo ln -s libboost_regex.so libboost_regex-mt.so
sudo ln -s libboost_regex.a libboost_regex-mt.a

...

Option 1 - Building Impala (for developing Impala)

This will also load all the test data, i.e. on HDFS, Kudu and HBase and for all formats (text/parquet/orc) and compressions (none/gzip/snappy etc.) which usually takes hours. If you just need some simple test data (e.g. TPCH data set), try Option 2 below.

git clone https://gitbox.apache.org/repos/asf/impala.git ~/Impala
cd ~/Impala
export IMPALA_HOME=`pwd`
bin/bootstrap_development.sh

Option 2 - Building Impala without Test Data (for manually testing Impala)

If you have some test data in hand, or you just need a subset of the test data (e.g. TPCH tables in parquet format), you can try this.

git clone https://gitbox.apache.org/repos/asf/impala.git ~/Impala
cd ~/Impala
export IMPALA_HOME=`pwd`
bin/bootstrap_system.sh
source ./bin/impala-config.sh
# Format the test cluster and start Impala and dependent services
./buildall.sh -noclean -notests -format -start_minicluster -start_impala_cluster

# Load some test data (skip this if you don't need them)
# Example 1: Load functional test data in parquet format. Note that you need the text tables since parquet tables are generated from them.
bin/create_testdata.sh
bin/load-data.py -e core -w functional-query --table_formats=text/none,parquet/none
# Example 2: Load TPCH test data in parquet format.
bin/load-data.py -e core -w tpch --table_formats=text/none,parquet/none
# Example 3: Load TPCDS test data in parquet format.
bin/load-data.py -e core -w tpcds --table_formats=text/none,parquet/none

Rebuilding after initial build

sudo# Rebuild yumboth groupinstallbackend "Developmentand Tools"
sudo yum -y install git ant libevent-devel automake libtool flex bison gcc-c++ openssl-devel make cmake doxygen.x86_64 glib-devel python-devel bzip2-devel svn libevent-devel krb5-workstation openldap-devel db4-devel python-setuptools python-pip cyrus-sasl* postgresql postgresql-server ant-nodeps lzo-devel lzop
sudo pip-python install allpairs pytest pytest-xdist paramiko texttable prettytable sqlparse psutil==0.7.1 pywebhdfs gitpython jenkinsapi boto3

##### Configuring Boost
Newer versions of boost need to be configured to work with Impala. Impala relies on certain boost libraries ending with a "-mt" extension. You need to add symlinks to boost_filesystem, boost_regex, and boost_system.

On CentOS 6.5

cd /usr/local/lib
sudo ln -s libboost_filesystem.so libboost_filesystem-mt.so
sudo ln -s libboost_filesystem.a libboost_filesystem-mt.a
sudo ln -s libboost_system.so libboost_system-mt.so
sudo ln -s libboost_system.a libboost_system-mt.a
sudo ln -s libboost_regex.so libboost_regex-mt.so
sudo ln -s libboost_regex.a libboost_regex-mt.a

##### Configuring Postgresql
If you are installing Impala on a fresh machine, you'll need to initialize postgres. On CentOs 6.5 this can be done by running

sudo service postgresql initdb

You need to make a configurations change to allow Hbase and the Hive metastore to functions correctly. Edit the following file as root.

...

On CentOs 6.5

/var/lib/pgsql/data/pg_hba.conf

In the following lines at the end of the file, change `peer` or `ident` to `trust`.

...

frontend
${IMPALA_HOME}/buildall.sh -skiptests -noclean

# Rebuild with optimized release binaries
${IMPALA_HOME}/buildall.sh -skiptests -noclean -release

# Incremental builds
source ${IMPALA_HOME}/bin/impala-config.sh # If you didn't already source impala-config.sh in this shell
 # Optional: Rebuild the impala binary only
make -j$IMPALA_BUILD_THREADS impalad
# Optional: Build the Java-side frontend only
make -j$IMPALA_BUILD_THREADS java

# Restart the Impala cluster
${IMPALA_HOME}/bin/start-impala-cluster.py

See Tips for Faster Impala Builds for more tips on how to do incremental builds.

FAQ

Q: The build is stuck at 100% progress for a while (as the screenshot shows). What's going on there?

Image Added

A: Usually the FE compilation is still running. Maven downloads a lot of artifacts. So you will see slow progress there if your internet connection is slow. You can check the logs at ${IMPALA_HOME}/logs/mvn/mvn.log

To make Postgres aware of these changes, either restart the service or run: pg_ctl reload

##### Creating the Hive metastore user
sudo -u postgres psql postgres

...

---

### Maven 3
wget http://www.interior-dsgn.com/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
tar xvf apache-maven-3.0.5-bin.tar.gz && sudo mv apache-maven-3.0.5 /usr/local

---

### Environment variables

Put these in your `.bashrc` or elsewhere:

On Ubuntu 14.04

export JAVA_HOME=/usr/lib/jvm/java-7-oracle
export IMPALA_HOME=<path to Impala>
export BOOST_LIBRARYDIR=/usr/lib/x86_64-linux-gnu
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
export LC_ALL="en_US.UTF-8"
export M2_HOME=/usr/local/apache-maven-3.0.5
export M2=$M2_HOME/bin
export PATH=$M2:$PATH

On CentOs 6.5

export BOOST_ROOT=/usr/local

---
### Add a path for HDFS domain sockets

sudo mkdir /var/lib/hadoop-hdfs/
sudo chown <user> /var/lib/hadoop-hdfs/
----
### Start local ssh server

sudo service ssh start

----
### Enable password-less SSH for HBase

ssh-keygen -t dsa
# Do not type in any passkey. Just press enter.
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

---
### Setup NTP for Kudu

On CentOS 7

yum install ntp
systemctl start ntpd
---