Use the right build command to build only the things that changed:
# Regenerate cmake files. ./buildall.sh -cmake_only # Rebuild impalad binary only: make -j ${IMPALA_BUILD_THREADS} impalad # Rebuild a backend test binary only: make -j ${IMPALA_BUILD_THREADS} buffered-block-mgr-test # Rebuild frontend only: make fe
Pass the right command line flags to skip unnecessary work:
- Add -notests to buildall.sh command to skip building backend tests
- Pass -noclean to buildall.sh if you don't need to do a full clean
Ccache - speeds up rebuilds when switching between branches
Ccache is a standard tool to accelerate C/C++ builds by caching previously-built object files. It can tremendously speed up builds when switching between branches of Impala.
# Install the ccache package (this example works on Ubuntu) sudo apt-get install ccache # Make sure you have a high limit on your max ccache size to ensure it is effective ccache --max-size=50G
Ninja - speeds up incremental builds
Ninja is an alternative to make that can resolve dependencies much faster. This can save 10+ seconds per invocation compared to make, which is significant for incremental builds (the difference in speed between ninja and make for full builds is minor compared to the end-to-end runtime of the build).
# Install the ninja package (this example works on Ubuntu) sudo apt-get install ninja-build # Instead of ./buildall.sh ...: ./buildall.sh ... -ninja # Instead of make -j ${IMPALA_BUILD_THREADS} <targets>: ninja -j ${IMPALA_BUILD_THREADS} <targets> # You can use an alias to make ninja use IMPALA_BUILD_THREADS by default alias ninja='ninja -j ${IMPALA_BUILD_THREADS}'
Skip toolchain bootstrapping
Bootstrapping the toolchain and python dependencies can add latency to buildall.sh invocations even if nothing needs to be downloaded. You can disable this by setting SKIP_TOOLCHAIN_BOOTSTRAP=true in bin/impala-config-local.sh. Note that this means you need to manually set SKIP_TOOLCHAIN_BOOTSTRAP=false when running buildall.sh to download dependencies when they do change.
Using distcc
The instructions for using distcc are in bin/distcc/README.md