...
- You want to build Impala purely from source without relying on prebuilt binaries.
- You are trying to build Impala on a platform for which prebuilt binaries are not available.
You want to add or modify a dependency in native-toolchain. E.g. build Impala with a newer version of LLVM, or with different configuration options.
Prerequisites
A number of prerequisites are required to build the full toolchain.
Below I have documented the requirements for Ubuntu 16.04 along with commands to install it. Package names and other details may vary between operating systems and versions. Also, if a package fails to build and you take steps to fix it, you might want to remove the old built directory in "$SOURCE_DIR/source/<package-name>/" before you retry building it again.
Code Block | ||
---|---|---|
| ||
# General build requirements sudo apt-get install build-essential git sudo apt-get install texinfo # Requirements for specific packages sudo apt-get install bison # For binutils sudo apt-get install autoconf automake libtool # For libevent sudo apt-get install libz-dev # For OpenSSL sudo apt-get install libssl-dev # For Thrift sudo apt-get install libncurses-dev # For GDB sudo apt-get install libsasl2-dev libkrb5-dev # For Kudu |
Building
Code Block | ||
---|---|---|
| ||
# Checkout the toolchain git clone https://github.com/cloudera/native-toolchain.git cd native-toolchain # This may take a long time! See check/ for detailed logs. ./buildall.sh |
Using custom toolchain with Impala
Starting with Impala 4.3.0, you can use a custom native toolchain with
Code Block |
---|
# Set a path for the toolchain. If this directory doesn't exist, Impala's buildall.sh script will download and build
# native-toolchain according to details in impala-config.sh.
export NATIVE_TOOLCHAIN_HOME=/path/to/native-toolchain
# Start the Impala build. This will (re)build native-toolchain and use the results in ${NATIVE_TOOLCHAIN_HOME}/build.
cd ${IMPALA_HOME}
. bin/impala-config.sh
./buildall.sh <args> |
For Impala checkouts prior to 4.3.0, use
Code Block | ||
---|---|---|
| ||
# Assume that ${IMPALA_HOME} has the Impala source checkout and ${NATIVE_TOOLCHAIN_HOME} has the native-toolchain source checkout cd ${IMPALA_HOME} # Symlink the built packages into the toolchain/ subdirectory (mkdir -p toolchain && cd toolchain && ln -s ${NATIVE_TOOLCHAIN_HOME}/build/* .) # Optional: ${IMPALA_HOME}/toolchain is the default value for IMPALA_TOOLCHAIN export IMPALA_TOOLCHAIN=Disable toolchain bootstrapping, which could replace our binaries with precompiled ones. # Adding this to impala-config-local.sh makes it permanent for this Impala checkout. echo "export SKIP_TOOLCHAIN_BOOTSTRAP=true" >> bin/impala-config-local.sh . bin/impala-config.sh # Optional: ${IMPALA_HOME}/toolchain # Disableis toolchainthe bootstrapping,default whichvalue could replace our binaries with precompiled onesfor IMPALA_TOOLCHAIN. You may want to add this to your .bashrc file. export SKIPIMPALA_TOOLCHAIN_BOOTSTRAP=true=${IMPALA_HOME}/toolchain # # Build Impala - this will pick up the new toolchain. ./buildall.sh <args> |
...