You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
We have Jenkins jobs able to build a Debian package in a cowbuilder image. The jobs use a serie of shell wrapper which streamline the environment setup and debian toolchain: http://jenkins-debian-glue.org/.
The jobs are running on dedicated Jenkins agents hosted on WMCS. They come with cowbuilder images provisioned by our Puppet modules
- create images for the distribution we care about (ex: trusty, jessie, unstable)
- auto update the images on a daily basis
- provides hook to support injecting apt.wikimedia.org components and the <release>-backports distribution (example: buster-backports).
For Gerrit repositories having Debian packages, the Jenkins job is registered in Zuul workflow to trigger:
- integration/config.git /zuul/layout.yaml
- name: operations/debs/contenttranslation/apertium test: - 'debian-glue'
It instructs CI to run the
debian-glue job for any patchset proposed to that Gerrit repository. But it only triggers if the change touches a file under the
debian directory. The job then:
- clones the repository
- checkouts the patch that has been merged by CI against the tip of the targeted branch
distributionto the distribution mentioned in
debian/changelog(see [#distributions] below).
- invokes Jenkins Debian Glue.
Jenkins Debian Glue will set
DIST based on
distribution. Then it triggers the build process in the matching cowbuilder image. When
pbuilder is run, the Wikimedia hooks are invoked and whenever the
debian/changelog distribution is suffixed with
-wikimedia, Wikimedia more specific components (eg: thirdparty) will be added.
Specific env variables can be injected by Zuul. This is done by altering a python script executed by Zuul whenever it triggers a job. That can be used to set
BACKPORTS which instruct our hooks to inject the release-backports components, tweak the build timeout with
BUILD_TIMEOUT or pass
DEB_BUILD_OPTIONS. Example as of May 2020:
- integration/config.git /zuul/parameter_functions.py
if 'debian-glue' in job.name: # XXX # When adding new paramters, make sure the env variable is added as an # env_keep in the sudo policy: # https://horizon.wikimedia.org/project/sudo/ # if 'nocheck' in job.name: params['DEB_BUILD_OPTIONS'] = 'nocheck' if 'backports' in job.name: # T173999 params['BACKPORTS'] = 'yes' # Always set the value to be safe (T144094) params['BUILD_TIMEOUT'] = 30 # minutes # Finely tweak jenkins-debian-glue parameters if params['ZUUL_PROJECT'] == 'integration/zuul': # Uses dh_virtualenv which needs access to pypy.python.org params['PBUILDER_USENETWORK'] = 'yes' elif (params['ZUUL_PROJECT'] == 'operations/debs/varnish4'): # VTC tests take forever params['BUILD_TIMEOUT'] = 60 # minutes params['DEB_BUILD_OPTIONS'] = 'parallel=12' elif (params['ZUUL_PROJECT'] == 'operations/software/varnish/varnishkafka'): # needed for librdkafka1 >= 0.11.5 params['BACKPORTS'] = 'yes' elif (params['ZUUL_PROJECT'] == 'operations/software/atskafka'): # needed by go build to access gopkg.in params['PBUILDER_USENETWORK'] = 'yes' elif (params['ZUUL_PROJECT'] == 'operations/debs/trafficserver'): # Building ATS takes a while params['BUILD_TIMEOUT'] = 60 # minutes # Backports needed on stretch for libbrotli-dev and a recent # debhelper version (>= 11) params['BACKPORTS'] = 'yes' elif (params['ZUUL_PROJECT'] == 'operations/debs/contenttranslation/giella-sme'): # Heavy build T143546 params['BUILD_TIMEOUT'] = 180 # minutes
Since the build is done with a
sudo cowbuilder, each new environment variable has to be whitelisted in the sudo policy of the
integration labs project.
Non exhaustive list as of September 2016:
||Distribution used by cowbuilder, pbuilder and Wikimedia pbuilder hook|
||Architecture (i386, amd64)|
||Debian glue setting|
||Debian glue setting. Set by the job to the distribution in |
||Base directory of the Jenkins job. Set by Jenkins.|
|| Defined by Debian Policy, let you change behavior of the build process if proper support is added in |
Jenkins debian glue parses the
debian/changelog to find the target distribution. When the changelog entry targets
UNRELEASED, CI picks the distribution from the previous changelog entry.
In some case ones want to set both
BACKPORTS, but both can not be set in the changelog entry. Our convention is to suffix the distribution with
-wikimedia in the changelog and then trigger the more specific job
debian-glue-backports which cause CI to set
Some repository want a single branch to support multiple distributions. We thus have a set of jobs that hardcode the distribution and do not extract it from
This is example for
labs/toollabs which in zuul/layout.yaml has:
- name: labs/toollabs test: # Single branch supporting multiple distributions T210780 - debian-glue-unstable - debian-glue-stretch - debian-glue-buster gate-and-submit: - debian-glue-unstable - debian-glue-stretch - debian-glue-buster
- Jenkins Debian Glue settings with documentation: http://jenkins-debian-glue.org/docs/
- Zuul workflow https://phabricator.wikimedia.org/diffusion/CICF/browse/master/zuul/layout.yaml
- Zuul python hook to inject env variables https://phabricator.wikimedia.org/diffusion/CICF/browse/master/zuul/parameter_functions.py
- Wikitech sudo policy: Special:NovaSudoer
- Definition of our Jenkins jobs using the Jenkins Job Builder DSL: https://phabricator.wikimedia.org/diffusion/CICF/browse/master/jjb/operations-debs.yaml