This is an attempt to document the minimum set of prerequisites to build and run Impala tests on Ubuntu 16.04.

I believe many of the prerequisites documented in various places aren't actually necessary since we moved to native-toolchain and various operating systems upgraded packages.

Related: Building native-toolchain from scratch and using with Impala

 

# General build requirements
sudo apt-get install build-essential git maven
# General Java requirements
sudo apt-get install openjdk-8-jdk
#==============#
# Impala Build #
#==============#
# Packages required to build Python virtualenv
sudo apt-get install libpython-dev
# Ubuntu 16.04 doesn't have /usr/bin/python symlink by default, which is required by some shebangs. Package "python" provides it. 
sudo apt-get install python
# Packages required to build Impala
sudo apt-get install libssl-dev libsasl2-dev libkrb5-dev

# Minimum environment variables required to build.
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64/

# Additional recommended variable to speed up builds.
export USE_GOLD_LINKER=true
./buildall.sh -noclean -notests

# TODO: wait for Ubuntu 16.04 fixes: std::isnan() and std::isinf().

#==================#
# Test environment #
#==================#
# Postgres
sudo apt-get install postgresql
sudo service postgresql start
# Edited pg_hba.conf to change "peer" to "trust"
sudo -u postgres psql postgres -c "CREATE ROLE hiveuser LOGIN PASSWORD 'password'; ALTER ROLE hiveuser WITH CREATEDB;"

# Setup passwordless ssh for hbase
sudo apt-get install openssh-server
sudo service ssh start
ssh-keygen -t dsa
# Do not type in any passkey. Just press enter.
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

# Setup ntpd for Kudu
sudo apt-get install ntp
sudo systemctl restart ntp.service

# Add a path for HDFS domain sockets
sudo mkdir /var/lib/hadoop-hdfs/
sudo chown <user> /var/lib/hadoop-hdfs/
 
# Get lzo libraries for data loading
git clone https://github.com/cloudera/impala-lzo.git ../Impala-lzo
git clone https://github.com/cloudera/hadoop-lzo.git ../hadoop-lzo
sudo apt-get install liblzo2-dev
cd ../hadoop-lzo
ant package
 
# Setup loopback # TODO

 

 

 

  • No labels

2 Comments

  1. In my opinion, this is now covered by bin/bootstrap_system.sh. Any objection to deleting it?

  2. Can we leave the page but replace the current contents with a pointer to the script? I think that makes it easier to figure out than hoping people will find the right part of the script to read.