This page documents how to build https://github.com/cloudera/native-toolchain from scratch and use it with Impala.
This is for advanced use only. It is usually easier to use prebuilt binary packages that are downloaded by Impala's bootstrap_toolchain.py script. This instructions are useful in a few scenarios:
- You want to build Impala purely from source without relying on prebuilt binaries.
- You are trying to build Impala on a platform for which prebuilt binaries are not available.
You want to add or modify a dependency in native-toolchain. E.g. build Impala with a newer version of LLVM, or with different configuration options.
Prerequisites
A number of prerequisites are required to build the full toolchain.
Below I have documented the requirements for Ubuntu 16.04 along with commands to install it. Package names and other details may vary between operating systems and versions.
# General build requirements sudo apt-get install build-essential git # Requirements for specific packages sudo apt-get install bison # For binutils sudo apt-get install autoconf automake libtool # For libevent sudo apt-get install libz-dev # For OpenSSL sudo apt-get install libssl-dev # For Thrift sudo apt-get install libncurses-dev # For GDB sudo apt-get install libsasl2-dev # For Kudu
Building
# Checkout the toolchain git clone https://github.com/cloudera/native-toolchain.git cd native-toolchain # This may take a long time! See check/ for detailed logs. ./buildall.sh
Using custom toolchain with Impala
# Assume that ${IMPALA_HOME} has the Impala source checkout and ${NATIVE_TOOLCHAIN_HOME} has the native-toolchain source checkout cd ${IMPALA_HOME} # Symlink the built packages into the toolchain/ subdirectory (mkdir -p toolchain && cd toolchain && ln -s ${NATIVE_TOOLCHAIN_HOME}/build/* .) # Disable toolchain bootstrapping, which could replace our binaries with precompiled ones. You may want to add this to your .bashrc file. export SKIP_TOOLCHAIN_BOOTSTRAP=true # Optional: ${IMPALA_HOME}/toolchain is the default value for IMPALA_TOOLCHAIN. You may want to add this to your .bashrc file. export IMPALA_TOOLCHAIN=${IMPALA_HOME}/toolchain # Build Impala - this will pick up the new toolchain. ./buildall.sh <args>