Mousebender

Those of you that followed Cheesecake buildbot may have noticed its strange behavior. Tests, that we knew should pass, failed with strange errors and the buidbot itself hang from time to time. I must confess it was my fault. Problems were caused by misconfiguration – I tried to run two instances of nose test runners at the same time. That way one instance of test get into way of another one and we were getting errors similar to:

OSError: [Errno 2] No such file or directory:
'/tmp/cheesecake_sandbox/nose-0.8.3.log'

First Cheesecake instance have created this file, but another one cleaned up global /tmp directory, thus the error. To fix this you have to run builds independently, one after another. At first, I used Periodic again, configuring it like this:

c['schedulers'] = [
    Periodic("every_6_hours+10min", ["x86_rh9_trunk"], 6*60*60+10*60),
    Periodic("every_6_hours", ["x86_rh9_mk"], 6*60*60)
]

But it won’t do what I wanted, not counting the first time. First object will run x86_rh9_trunk build every 6 hours and 10 minutes, second will run x86_rh9_mk build every 6 hours. That means that the interval between these two build will rise in time. First it will be 10 minutes, then 20 minutes, then half an hours, etc. That’s not really what I wanted. Searching a buildbot mailing list I’ve found a thread about a cron-style scheduler, that is capable of running builds at certain points in time. Implementation of this scheduler was merged into main buildbot development line with a name Nightly. I was happy to see that it was finally the module I was looking for. My configuration looks like this:

c['schedulers'] = [
    Nightly("every_6_hours", ["x86_rh9_mk"], hour=[9,15,21,3],
             minute=0),
    Nightly("every_6_hours+10min", ["x86_rh9_trunk"],
             hour=[9,15,21,3], minute=10),
]

It will run x86_rh9_mk build at 9.00am, 3.00pm, 9.00pm and 3.00am (so every 6 hours) and x86_rh9_trunk build at 9.10am, 3.10pm, 9.10pm and 3.10am (so every 6 hours, always 10 minutes later than x86_rh9_mk). buildbot seems to be configured much better now, so I can focus on coding exclusively.

I must admin setting up a buildbot was very entertaining. Of course this is because I read Grig tutorial. With well-written HOWTO nothing is hard. I’ll just write down few things that wasn’t already mentioned by Grig.

Oh, and if you can’t wait, Cheesecake buildbot is available here.

installation

Before we can install buildbot, few prerequisites have to be met. First, Zope Interface. I’ve chosen stable 3.0.1 version. After downloading and unpacking, I’ve tried:

# python setup.py install

This unfortunately failed with error:

error: static declaration of ‘SpecType’ follows non-static declaration

With minimal googling around I’ve managed to find a fix. Bug has been described on Zope issue tracker. Trivial fix is also available for download.

$ cd Dependencies/zope.interface-ZopeInterface-3.0.1/\
zope.interface/
$ patch -p4 < fix-zope-compilation.patch
patching file _zope_interface_coptimizations.c

After that I could try my luck again with setup.py install. Installation was successfull this time. Next prerequisite is Twisted and TwistedWeb packages. I’ve downloaded a TwistedSumo tarball and installed Twisted-2.2.0 and TwistedWeb-0.5.0 with the help of almighty setup.py install command.

Installing last package is pretty easy. You just have to execute one command:

# easy_install http://twistedmatrix.com/users/acapnotic/wares/code/CVSToys/CVSToys-1.0.9.tar.bz2

If you have any problems with easy_install (it may throw error Not a recognized archive type at you on some systems) you may also download, unpack and install the package manually. easy_install just automates this process.

Builbot 0.7.3 was realeased this Tuesday. You can grab any version at sourceforge. Install with standard python setup.py install. You should get buildbot in /usr/bin/ installed.

buildmaster

To set up a buildmaster few steps have to be followed. First create a separate user:

# adduser buildmaster

Now su to this newly created account and execute these commands:

$ mkdir conf
$ buildbot master conf/
$ cd conf/
$ mv master.cfg.sample master.cfg

In conf/master.cfg you will get a configuration template you should modify to get appropriate results. My complete master.cfg file follow (without comments).

import os.path
from buildbot.changes.freshcvs import FreshCVSSource
from buildbot.scheduler import Scheduler, Periodic
from buildbot.process import step, factory
from buildbot.status import html
s = factory.s

c = BuildmasterConfig = {}

c['bots'] = [("x86_rh9", "*PASSWORD*")]

c['sources'] = []

c['schedulers'] = [
    Periodic("every_6_hours", ["x86_rh9_trunk"], 6*60*60),
    Periodic("every_6_hours2", ["x86_rh9_mk"], 6*60*60)
]

make_source = lambda branch: s(step.SVN, mode='update',
                baseURL='http://svn.pycheesecake.org/',
                defaultBranch=branch)

source_mk = make_source('branches/mk/')
source_trunk = make_source('trunk')

class StepTest(step.ShellCommand):
    name = "nose tests"
    description = ["running nose tests"]
    descriptionDone = [name]

nose_tests = s(StepTest, command="/usr/bin/python setup.py test")

make_factory = lambda source: factory.BuildFactory([source, nose_tests])

f_mk = make_factory(source_mk)
f_trunk = make_factory(source_trunk)

c['builders'] = [
    {'name':'x86_rh9_mk',
    'slavename':'x86_rh9',
    'builddir':'test-mk',
    'factory':f_mk
    },
    {'name': 'x86_rh9_trunk',
    'slavename': 'x86_rh9',
    'builddir': 'test-trunk',
    'factory': f_trunk
    },
]

c['slavePortnum'] = 9999

c['status'] = []
c['status'].append(html.Waterfall(http_port=8888))

c['projectName'] = "Cheesecake"
c['projectURL'] = "http://pycheesecake.org/"

c['buildbotURL'] = "http://agilistas.org:8888"

Note that you should set up password yourself.

buildslave

After buildmaster setup is complete, we can configure build slaves, which is far easier as no configuration files come into play. Simply add a slave user:

# adduser buildslave

and make a directory:

$ mkdir conf
$ buildbot slave /home/buildslave/conf localhost:9999 x86_rh9 *PASSWORD*

Note that the port given here and c[‘slavePortnum’] in master configuration must match.

running

To start a build slave, run following command as user buildslave:

buildbot start /home/buildslave/conf

Then start a build master with simillar command (run as user buildmaster):

buildbot start /home/buildmaster/conf

That’s it! You can now access your buildbot through web interface at an address you’ve specified in c[‘buildbotURL’] configuration option. If this doesn’t work check out logs (twistd.log) in your slave and master directories.

References

• BuildBot Manual
• Continuous integration with buildbot