...
Code Block | ||
---|---|---|
| ||
# Install the ccache package (this example works on Ubuntu) sudo apt-get install ccache # Make sure you have a high limit on your max ccache size to ensure it is effective ccache --max-size=50G |
Note: If you work on the same machine locally and remotely, make sure you use the same environment. ssh client can send through environment variables (and will do it by default for some, e.g. $LC_CTYPE), which is included by ccache for hashing and causes a cache miss even if the environment variable does not affect the build and the artifact is already cached.
For more information how to control the sent/received environment variables with ssh config: https://stackoverflow.com/questions/29609371/how-not-to-pass-the-locale-through-an-ssh-connection-command
Ninja - speeds up incremental builds
...
Code Block | ||
---|---|---|
| ||
# Install the ninja package (this example works on Ubuntu) sudo apt-get install ninja-build # Instead of ./buildall.sh ...: ./buildall.sh ... -ninja # Instead of make -j ${IMPALA_BUILD_THREADS} <targets>: ninja -j ${IMPALA_BUILD_THREADS} <targets> # You can use an alias to make ninja use IMPALA_BUILD_THREADS by default alias ninja='ninja -j ${IMPALA_BUILD_THREADS}' |
Using distcc
The instructions for using distcc are in bin/distcc/README.md
Skip checking toolchain
...
dependencies (advanced only)
By default buildall.sh will check for Bootstrapping the toolchain and python dependencies package updates, which can add latency some time to buildall.sh invocations even if nothing needs to be downloaded. You can disable this by setting SKIP_TOOLCHAIN_BOOTSTRAP=true in adding this line to bin/impala-config-local.sh. Note that this means you need to manually set :
Code Block | ||
---|---|---|
| ||
export SKIP_TOOLCHAIN_BOOTSTRAP= |
...
true |
If you do this then you may run into build failures because of missing dependencies, and you will need to temporarily disable this optimisation or directly download the dependencies (e.g. by running bin/bootstrap_toolchain.py)
Using distcc
The instructions for using distcc are in bin/distcc/README.md