You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org
Toolsadmin.wikimedia.org/Build
The Striker build and deploy process involves updating the labs-striker-deploy git repository and then pushing the repository out to servers running Striker via scap3.
Deploy directory structure
/srv/deployment/striker/deploy/ ├── _build/ # Generated by make_wheels.sh, only on Cloud VPS staging server | └── venv/ # virtualenv used for creating wheels & top level ├── ddl/ # Manually created SQL patches ├── make_wheels.sh # Build script for preparing labs-striker-wheels patches ├── public_html/ # Apache/nginx docroot | └── static/ # labs-striker-staticfiles submodule ├── requirements.txt # `pip --freeze` generated requirements ├── scap/ | ├── checks/ | | └── virtualenv.sh # Check script to rebuild venv on target hosts | ├── checks.yaml # Post-deploy checks for Scap3 | ├── dsh/ # Scap3 target host sets | ├── log/ # Generated by scap3, only on deploy master | └── scap.cfg # Scap3 config file ├── striker/ # striker submodule | └── requirements.txt # Product level requirements └── wheels/ # labs-striker-wheels submodule
Submodules
Most of the interesting code changes will happen in submodules of labs-striker-deploy:
- labs-striker: The Striker Django application
- labs-striker-staticfiles: Django managed static files (css, js, images)
- labs-striker-wheels: Python binary packages in wheel format
labs-striker
There is nothing special to do to prepare this repo for deployment. Merge commits using Gerrit.
Changes made here however will trigger changes to other submodules. The the point a release is being made:
- labs-striker-staticfiles must be updated if any changes were made to requirements.txt or if files were modified in the
static
directory. - labs-striker-wheels must be updated if any changes were made to requirements.txt.
labs-striker-staticfiles
These static assets are collected from the installed Django applications using Django's collectstatic
management script. The contrib/collectstatic.sh
convenience script is provided in labs-striker for collecting the files. This can be run in a MediaWiki-Vagrant development environment.
$ cd /vagrant/srv/striker
$ contrib/collectstatic.sh
Deleting 'robots.f71d20196d4c.txt'
Deleting 'robots.txt'
Deleting 'staticfiles.json'
...
Post-processed 'robots.txt' as 'robots.f71d20196d4c.txt'
110 static files copied to '/vagrant/srv/striker/staticfiles', 110 post-processed.
$ cd staticfiles
$ python -mjson.tool staticfiles.json > staticfiles.json.pretty
$ mv staticfiles.json.pretty staticfiles.json
$ cp -R . $PATH_TO_LAB_STRIKER_STATICFILES
$ cd $PATH_TO_LAB_STRIKER_STATICFILES
$ git status
$ git add .
$ git commit
$ git review
Before submitting you may want to look for old assets that can be removed as well. These would generally be <FILENAME>.<CONTENT_HASH>.<EXT>
files that are for versions of the files that were removed from the manifest in a prior update that has already been deployed.
Testing in Cloud VPS project
TODO: explain how I typically stage pending changes in the striker Cloud VPS project.
Deploy
TODO: explain basic scap3 deployment