You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
We have Jenkins jobs able to build a Debian package in a cowbuilder image. The job uses a serie of shell wrapper which streamline the environment setup and debian toolchain: http://jenkins-debian-glue.org/.
As of September 2016, the jobs are running on Jessie jenkins agents. cobuilder images are provisioned by our Puppet modules
- create images for the distribution we care about (ex: trusty, jessie, unstable)
- auto update the images on a daily basis
- provides hook to support injecting apt.wikimedia.org components and the jessie-backports
For Gerrit repositories having Debian packages, the Jenkins job is registered in Zuul workflow to trigger:
- integration/config.git /zuul/layout.yaml
- name: operations/debs/contenttranslation/apertium test: - 'debian-glue'
Will cause any patchset proposed to that Gerrit repository to trigger the Jenkins job
debian-glue. The job:
- clone the repository
- checkout the patch merged against tip of the branch
distributionto the distribution mentioned in
- invoke Jenkins Debian Glue.
Jenkins Debian Glue will set
DIST based on
distribution. Then trigger the build process in the matching cowbuilder image. When pbuilder is run, the Wikimedia hooks are invoked and whenever the
debian/changelog distribution is suffixed with
-wikimedia, Wikimedia more specific components (eg: thirdparty) will be added.
As of September 2016, setting
BACKPORTS to inject the Ubuntu updates component or Debian backports components is NOT SUPPORTED.
Specific env variables can be injected by Zuul. This is done by altering a python script executed by Zuul whenever it triggers a job. Example as of September 2016:
- integration/config.git /zuul/parameter_functions.py
if 'debian-glue' in job.name: # Always set the value to be safe (T144094) params['BUILD_TIMEOUT'] = 30 # minutes # Finely tweak jenkins-debian-glue parameters if params['ZUUL_PROJECT'] == 'integration/zuul': # Uses dh_virtualenv which needs access to pypy.python.org params['PBUILDER_USENETWORK'] = 'yes' params['DEB_BUILD_OPTIONS'] = 'nocheck' elif (params['ZUUL_PROJECT'] == 'operations/debs/contenttranslation/giella-sme'): # Heavy build T143546 params['BUILD_TIMEOUT'] = 180 # minutes
Since the build is done with a
sudo cowbuilder, each new environment variable has to be whitelisted in the sudo policy of the
integration labs project.
Non exhaustive list as of September 2016:
||Distribution used by cowbuilder, pbuilder and Wikimedia pbuilder hook|
||Architecture (i386, amd64)|
||Debian glue setting|
||Debian glue setting. Set by the job to the distribution in |
||Base directory of the Jenkins job. Set by Jenkins.|
|| Defined by Debian Policy, let you change behavior of the build process if proper support is added in |
- Jenkins Debian Glue settings with documentation: http://jenkins-debian-glue.org/docs/
- Zuul workflow https://phabricator.wikimedia.org/diffusion/CICF/browse/master/zuul/layout.yaml
- Zuul python hook to inject env variables https://phabricator.wikimedia.org/diffusion/CICF/browse/master/zuul/parameter_functions.py
- Wikitech sudo policy: Special:NovaSudoer
- Definition of our Jenkins jobs using the Jenkins Job Builder DSL: https://phabricator.wikimedia.org/diffusion/CICF/browse/master/jjb/operations-debs.yaml