You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

This page documents how to build https://github.com/cloudera/native-toolchain from scratch and use it with Impala.

This is for advanced use only. It is usually easier to use prebuilt binary packages that are downloaded by Impala's bootstrap_toolchain.py script. This instructions are useful in a few scenarios:

  • You want to build Impala purely from source without relying on prebuilt binaries.
  • You are trying to build Impala on a platform for which prebuilt binaries are not available.

  • You want to add or modify a dependency in native-toolchain. E.g. build Impala with a newer version of LLVM, or with different configuration options.

Prerequisites

A number of prerequisites are required to build the full toolchain.

Below I have documented the requirements for Ubuntu 16.04 along with commands to install it. Package names and other details may vary between operating systems and versions.

# General build requirements
sudo apt-get install build-essential git

# Requirements for specific packages
sudo apt-get install bison # For binutils
sudo apt-get install autoconf automake libtool # For libevent
sudo apt-get install libz-dev # For OpenSSL
sudo apt-get install libssl-dev # For Thrift
sudo apt-get install libncurses-dev # For GDB
sudo apt-get install libsasl2-dev # For Kudu

Building

# Checkout the toolchain
git clone https://github.com/cloudera/native-toolchain.git
cd native-toolchain

# This may take a long time! See check/ for detailed logs.
./buildall.sh

Using custom toolchain with Impala

# Assume that ${IMPALA_HOME} has the Impala source checkout and ${NATIVE_TOOLCHAIN_HOME} has the native-toolchain source checkout
cd ${IMPALA_HOME}
# Symlink the built packages into the toolchain/ subdirectory
(mkdir -p toolchain && cd toolchain && ln -s ${NATIVE_TOOLCHAIN_HOME}/build/* .)

# Disable toolchain bootstrapping, which could replace our binaries with precompiled ones. You may want to add this to your .bashrc file.
export SKIP_TOOLCHAIN_BOOTSTRAP=true
# Optional: ${IMPALA_HOME}/toolchain is the default value for IMPALA_TOOLCHAIN. You may want to add this to your .bashrc file.
export IMPALA_TOOLCHAIN=${IMPALA_HOME}/toolchain

 # Build Impala - this will pick up the new toolchain.
./buildall.sh <args>
  • No labels