Toolsadmin.wikimedia.org/Build

The Striker build and deploy process involves updating the labs-striker-deploy git repository and then pushing the repository out to servers running Striker via scap3.

Deploy directory structure

/srv/deployment/striker/deploy/
 ├── _build/                    # Generated by make_wheels.sh, only on Cloud VPS staging server
 |    └── venv/                 # virtualenv used for creating wheels & top level 
 ├── ddl/                       # Manually created SQL patches
 ├── make_wheels.sh             # Build script for preparing labs-striker-wheels patches
 ├── public_html/               # Apache/nginx docroot
 |    └── static/               # labs-striker-staticfiles submodule
 ├── requirements.txt           # `pip --freeze` generated requirements
 ├── scap/
 |    ├── checks/
 |    |    └──  virtualenv.sh   # Check script to rebuild venv on target hosts
 |    ├── checks.yaml           # Post-deploy checks for Scap3
 |    ├── dsh/                  # Scap3 target host sets
 |    ├── log/                  # Generated by scap3, only on deploy master
 |    └── scap.cfg              # Scap3 config file
 ├── striker/                   # striker submodule
 |    └── requirements.txt      # Product level requirements
 └── wheels/                    # labs-striker-wheels submodule

Submodules

Most of the interesting code changes will happen in submodules of labs-striker-deploy:

• labs-striker: The Striker Django application
• labs-striker-staticfiles: Django managed static files (css, js, images)
• labs-striker-wheels: Python binary packages in wheel format

labs-striker

There is nothing special to do to prepare this repo for deployment. Merge commits using Gerrit.

Changes made here however will trigger changes to other submodules. The the point a release is being made:

• labs-striker-staticfiles must be updated if any changes were made to requirements.txt or if files were modified in the static directory.
• labs-striker-wheels must be updated if any changes were made to requirements.txt.

labs-striker-staticfiles

These static assets are collected from the installed Django applications using Django's collectstatic management script. The contrib/collectstatic.sh convenience script is provided in labs-striker for collecting the files. This can be run in a MediaWiki-Vagrant development environment.

$ cd /vagrant/srv/striker
$ contrib/collectstatic.sh
Deleting 'robots.f71d20196d4c.txt'
Deleting 'robots.txt'
Deleting 'staticfiles.json'
...
Post-processed 'robots.txt' as 'robots.f71d20196d4c.txt'

110 static files copied to '/vagrant/srv/striker/staticfiles', 110 post-processed.
$ cd staticfiles
$ python -mjson.tool staticfiles.json > staticfiles.json.pretty
$ mv staticfiles.json.pretty staticfiles.json
$ cp -R . $PATH_TO_LAB_STRIKER_STATICFILES
$ cd $PATH_TO_LAB_STRIKER_STATICFILES
$ git status
$ git add .
$ git commit
$ git review

Before submitting you may want to look for old assets that can be removed as well. These would generally be <FILENAME>.<CONTENT_HASH>.<EXT> files that are for versions of the files that were removed from the manifest in a prior update that has already been deployed.

Testing in Cloud VPS project

TODO: explain how I typically stage pending changes in the striker Cloud VPS project.

Deploy

TODO: explain basic scap3 deployment