Continuous Testing

Wiki MarkupWe're running an experimental \[running a Jenkins Continuous Integration (CI) server at http://buildbotbuilds.sourceforgeapache.net/ BuildBot\] server; after every check-in to SVN, several [BuildBot] clients will automatically download the latest code and run "make test". The results \[http://buildbot.spamassassin.org:8010/ are visible here\].

Notes on Setting Up a BuildBot Slave

We currently have build slaves building

• trunk using Solaris 10 with perl 5.6.1 and perl 5.8.5
  

If you fancy it, and are running an OS different from the above, it might be worthwhile setting up a build slave to extend this... non-Solaris platforms especially would be great. "make test" needs to be passing on this platform, though, as otherwise we'll be bombarded with "test failure" messages, even if the tree builds on other OSes.

Notes:

For now, only committers' machines will be added to the BuildBot slaves list – we need to be sure that you're contactable, and we're not sure if there's any security issues involved!

There may be a possibility that if someone gets malicious code into SVN trunk at any time, it'll be run on your machine. So if that's worrying, best to avoid setting up a 'bot.

The slave process should be kept up and running as much as possible; it's got to be a persistent daemon.

I'd recommend running as non-root, and not as your own userid. if a miscreant managed to get hostile code into SVN trunk, it'd pretty quickly get run on your machine by this code.

It's not *too* CPU hungry – but will kick off a compile and "make test" *every time* someone checks something into SpamAssassin svn! so if that puts you off, this isn't for you

so pretty much, overall, this requires that you have root on some box which has a 99%-uptime network connection to set a slave up.

Commands To Set Up a BuildBot Slave

Install Twisted. This step can be omitted if you already have it, or just use "sudo apt-get install twisted" if you're on debian unstable or "make install" from the FreeBSD port.

Note that you also need python 2.2 or later installed.

cd Twisted-2.0.N ; sudo python setup.py install
cd ..

Next, install BuildBot itself:

tar xvfz buildbot-0.7.5.tar.gz
cd buildbot-0.7.5 ; sudo python setup.py install

And create a user account to run the slave as:

sudo useradd -c "SpamAssassin Buildbot" buildbot
sudo su - buildbot
mkdir -p /home/buildbot/slaves

Commands to create a new buildbot slave

You need a buildbot password be set up for you – a random string in a file in
/home/buildbot/pwds/. ask on the IRC channel and one of the PMC should be able to set you up with one.

PASSWORD=[password]
NAME=t-sol10-561

sudo mkdir -p /home/buildbot/slaves/$NAME
sudo chown buildbot /home/buildbot/slaves/$NAME
cd /home/buildbot/slaves/$NAME
sudo su buildbot -c "buildbot create-slave --usepty=0 \
         /home/buildbot/slaves/$NAME \
         buildbot.spamassassin.org:9989 $NAME $PASSWORD"

on the zone:

echo $PASSWORD > $HOME/pwd
sudo mv $HOME/pwd /home/buildbot/pwds/$NAME
sudo chown buildbot /home/buildbot/pwds/$NAME
sudo chmod 600 /home/buildbot/pwds/$NAME

sudo vi /home/buildbot/bots/buildbot/master.cfg

        [add new lines/entries for $NAME]
        [ensure you update 'schedulers' too!]

sudo vi /etc/init.d/buildbot 

        [add new lines/entries for $NAME, if it's running on the zone]

Command to start the slave process:

buildbot start /home/buildbot/slaves/$HOST_OS

To monitor slave progress/errors:

less /home/buildbot/slaves/$HOST_OS/twistd.log

To start at boot in future, add this line to crontab (on linux machines):

org/.

Due to politics with Oracle, the project formerly known as Hudson was renamed Jenkins in early 2011. http://jenkins-ci.org/content/hudsons-future

Therefore, the Jenkins/Hudson server for the SA Project has a bit of an identity crisis. This is evident in documentation, server names, existing scripts, etc. But the Jenkins CI server will run build/hudson/run_build (3.3 branch) or build/jenkins/run_build (trunk) to test the latest code after every check-in to SVN.

As of May 2011, there are jobs for both the 3.3 branch and trunk:

https://builds.apache.org/job/SpamAssassin-3.3.x/ https://builds.apache.org/job/SpamAssassin-trunk/

Both of these jobs run on the solaris1 slave and use the script in trunk under build/jenkins/run_build

There is also a test job we are using to prepare for the eventual replacement of the solaris1 zone for Jenkins.

https://builds.apache.org/job/SpamAssassin-trunk-FreeBSD/

This job runs on a freebsd slave and use the script in build/jenkins/run_build_freebsd

To login at http://builds.apache.org/, use your ASF LDAP credentials.

To administer the jobs, you must be a member of hudson-jobadmin group.

For more information, see http://wiki.apache.org/general/Jenkins which includes instructions for the PMC chair to grant access for a committer to get an administrative account.

The results can be viewed at the same links above for the jobs.

...