Buildbot Local¶
This example is for Debian based OS’s. You will need Python3 venv installed and a few other libraries.
sudo apt install python3-venv
sudo apt-get install build-essential python3-dev libssl-dev libffi-dev
Create the Master¶
In a terminal do the following commands.
mkdir -p ~/buildbot/master
cd ~/buildbot/master
python3 -m venv sandbox
source sandbox/bin/activate
pip install --upgrade pip
pip install 'buildbot[bundle]'
buildbot create-master charles
There is a sample master.cfg file but it’s less confusing to just create our own master.cfg. In the /buildbot/master/charles directory create an empty file named master.cfg.
master.cfg
# -*- python -*-
# ex: set filetype=python:
# import the needed plugins
from buildbot.plugins import worker, util, schedulers, reporters, changes, steps
# This is the dictionary that the buildmaster pays attention to. We also use
# a shorter alias to save typing.
c = BuildmasterConfig = {}
'''
WORKERS
The 'workers' list defines the set of recognized workers. Each element is
a Worker object, specifying a unique worker name and password. The same
worker name and password must be configured on the worker. This is Python so
the [] denotes that it's a list so multiple workers are seperated by a comma.
for example
c['workers'] = [worker.Worker('bee1', 'pass'), worker.Worker('bee2', 'pass')]
'''
c['workers'] = [worker.Worker('workerbee', 'pass')]
'''
PROTOCOLS
protocols contains information about protocols which master will use for
communicating with workers. You must define at least 'port' option that workers
could connect to your master with this protocol.
'port' must match the value configured into the workers (with their
--master option)
'''
c['protocols'] = {'pb': {'port': 9989}}
'''
CHANGE SOURCES
the 'change_source' setting tells the buildmaster how it should find out
about source code changes. Here we point to the bbtest project. You should point
to a project of your own so you can test changes. The change_source object is
a list so you can .append more changes to the list. The branch is the branch to
check, the project is the name we will assoiate with a scheduler. The
pollInterval is how many seconds to wait between checking the repo for changes
with the GitPoller object. This example is for a github.com repo, the offical
docs have info about other Version Control Systems.
'''
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
'https://github.com/jethornton/bbtest.git', branch='master', project='bbtest',
pollInterval=120))
'''
SCHEDULERS
The Schedulers decide how to react to incoming changes. A SingleBranchScheduler
checks one branch. The project is a filter used by the schedulers to know what
to react to. When the treeStableTimer is set to None the scheduler will react
to every change in the repo. The builderNames is a list of builders.
'''
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(name="buildbb",
change_filter=util.ChangeFilter(project='bbtest', branch='master'),
treeStableTimer=None, builderNames=['buildtest']))
'''
BUILDERS
The 'builders' list defines the Builders, which tell Buildbot how to perform a
build, what steps to take, and which workers can execute them. Note that any
particular build will only take place on one worker.
'''
c['builders'] = []
'''
First we must create a BuildFactory and we name this one bf. Next we add the
steps to take. The first step updates the local copy of the repo located in
~/buildbot/worker/worker1/buildtest/build. In my example repo it's setup to
build a deb so the second step does that. Note the workdir is the place to
execute the debuild command.
'''
bf = util.BuildFactory()
bf.addStep(steps.Git(repourl='https://github.com/jethornton/bbtest.git', mode='incremental'))
bf.addStep(steps.ShellCommand(workdir='build/bbtest', command=["debuild", "-us", "-uc"]))
c['builders'].append(
util.BuilderConfig(name="buildtest", workernames=["workerbee"], factory=bf))
'''
BUILDBOT SERVICES
The services is a list of BuildbotService items like reporter targets. The
status of each build will be pushed to these targets. This example creates a
bot on the irc in the #bbot channel. The authz '' is any safe operation, '!' is
force, stop, shutdown. In order to allow anyone to force a build we define force
as true. The authorization is purely nick-based, so it only makes sense if the
specified nicks are registered to the IRC server. You can't use a grouped nick.
'''
c['services'] = []
c['services'].append(reporters.IRC(host="irc.libera.chat", nick="reporter",
authz={'': True, 'force': True, ('stop', 'shutdown'): ['fred-flintstone']},
notify_events=['start', 'finish'], channels=[{"channel": "#bbot"}]))
'''
PROJECT IDENTITY
The 'title' string will appear at the top of this buildbot installation's
home pages (linked to the 'titleURL').
'''
c['title'] = "JT Builder"
c['titleURL'] = "https://github.com/jethornton/bbtest.git/"
'''
the 'buildbotURL' string should point to the location where the buildbot's
internal web server is visible. This typically uses the port number set in
the 'www' entry below, but with an externally-visible host name which the
buildbot cannot figure out without some help. In this example the buildbot will
be on http://localhost:8010/
'''
c['buildbotURL'] = "http://localhost:8010/"
'''
A minimalistic config to activate new web UI with a waterfall view and a
console view and a grid view.
'''
c['www'] = dict(port=8010, plugins=dict(waterfall_view={}, console_view={},
grid_view={}))
'''
DB URL
This specifies what database buildbot uses to store its state.
It's easy to start with sqlite, but it's recommended to switch to a dedicated
database, such as PostgreSQL or MySQL, for use in production environments.
http://docs.buildbot.net/current/manual/configuration/global.html#database-specification
'''
c['db'] = {'db_url' : "sqlite:///state.sqlite",}
'''
BUILDBOT NET USAGE DATA
This turns off the buildbot reporting usage data to buildbot.org
'''
c['buildbotNetUsageData'] = None
master.cfg without all the comments
# -*- python -*-
# ex: set filetype=python:
from buildbot.plugins import worker, util, schedulers, reporters, changes, steps
c = BuildmasterConfig = {}
c['workers'] = [worker.Worker('workerbee', 'pass')]
c['protocols'] = {'pb': {'port': 9989}}
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
'https://github.com/jethornton/bbtest.git', branch='master', project='bbtest',
pollInterval=120))
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(name="buildbb",
change_filter=util.ChangeFilter(project='bbtest', branch='master'),
treeStableTimer=None, builderNames=['buildtest']))
c['builders'] = []
bf = util.BuildFactory()
bf.addStep(steps.Git(repourl='https://github.com/jethornton/bbtest.git', mode='incremental'))
bf.addStep(steps.ShellCommand(workdir='build/bbtest', command=["debuild", "-us", "-uc"]))
c['builders'].append(
util.BuilderConfig(name="buildtest", workernames=["workerbee"], factory=bf))
c['services'] = []
c['services'].append(reporters.IRC(host="irc.libera.chat", nick="reporter",
authz={'': True, 'force': True, ('stop', 'shutdown'): ['fred-flintstone']},
notify_events=['start', 'finish'], channels=[{"channel": "#bbot"}]))
c['title'] = "JT Builder"
c['titleURL'] = "https://github.com/jethornton/bbtest.git/"
c['buildbotURL'] = "http://localhost:8010/"
c['www'] = dict(port=8010, plugins=dict(waterfall_view={}, console_view={},
grid_view={}))
c['db'] = {'db_url' : "sqlite:///state.sqlite",}
c['buildbotNetUsageData'] = None
After creating the master.cfg file check it for errors with the following command.
buildbot checkconfig charles/master.cfg
Now we can start the master charles with this command
buildbot start charles
In a seperate terminal we can create the worker
mkdir -p ~/buildbot/worker
cd ~/buildbot/worker
python3 -m venv sandbox
source sandbox/bin/activate
pip install --upgrade pip
pip install buildbot-worker
buildbot-worker create-worker worker1 localhost workerbee pass
Now you can start the worker with
buildbot-worker start worker1
Now in a browser go to http://localhost:8010/ to view the buildbot web page.