You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

This document is for you to quickly setup CI jobs to ensure the quality of your own customized Bigtop distribution.

Notice that, the document assume you already know well how to use Jenkins, hence the instructions are brief.

If you have questions, feel free to ask on Bigtop mailing list. (smile)

Setup a Jenkins master

If you already have a managed Jenkins/Hudson, or you prefer your all installation, go to Setup Bigtop packages build matrix

To setup a Jenkins master by leveraging Docker, do the following:

# create jenkins user on host machine with uid=1000 to map the jenkins uid inside jenkins image
sudo adduser jenkins -u 1000
sudo yum install -y docker git
sudo su - jenkins -c "git config --global user.email \"jenkins@bigtop.apache.org\""
sudo su - jenkins -c "git config --global user.name \"jenkins\""
sudo usermod -a -G docker jenkins
sudo service docker start
sudo su - jenkins -c "docker run -d --name jenkins-master -p 80:8080 -v `pwd`:/var/jenkins_home jenkins"

And the needed Plugin(s):

  • git plugin

Setup Bigtop packages build matrix

Create a new job:

New Item -> Multi-configuration project

In Source Code Management section, fill in your repo and the branch, for example:

Repository URL: https://git-wip-us.apache.org/repos/asf/bigtop.gitBranch Specifier: master

In the Configuration Matrix section, add a user defined axis with following name and values:

Name: BUILD_ENVIRONMENTSValues: 
centos-6
centos-7
fedora-20
ubuntu-14.04
debian-8
opensuse-13.2

Add another user defined axis:

Name: COMPONENTS


Values: 
bigtop-groovy
bigtop-jsvc
bigtop-tomcat
bigtop-utils
crunch
datafu
flume
giraph
hadoop
hama
hbase
hive
hue
ignite-hadoop
kafka
kite
mahout
oozie
phoenix
pig
solr
spark
sqoop
sqoop2
tachyon
tez
ycsb
zookeeper

Finally, add a shell build step with the following script:

docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \
bash -l -c 'A=rpm ; type -p dpkg && A=deb ; ./gradlew allclean ; ./gradlew -Dbuildnodeps=true $COMPONENTS-$A'

The result will be looked like this:

http://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/

However, do aware that full matrix build of packages is time consuming, and they need roughly 50GB disk space for each build. 

Setup Bigtop packages build matrix


  • No labels