You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Toolsadmin.wikimedia.org/Build: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>BryanDavis
No edit summary
imported>BryanDavis
(Add an overview of the deploy directory structure)
Line 1: Line 1:
The [[Striker]] '''build and deploy process''' involves updating the [[phab:diffusion/LSTD/|labs-striker-deploy]] git repository and then pushing the repository out to servers running Striker via [[scap3]].
The [[Striker]] '''build and deploy process''' involves updating the [[phab:diffusion/LSTD/|labs-striker-deploy]] git repository and then pushing the repository out to servers running Striker via [[scap3]].
== Deploy directory structure ==
<pre>
/srv/deployment/striker/deploy/
├── _build/                    # Generated by make_wheels.sh, only on Cloud VPS staging server
|    └── venv/                # virtualenv used for creating wheels & top level
├── ddl/                      # Manually created SQL patches
├── make_wheels.sh            # Build script for preparing labs-striker-wheels patches
├── public_html/              # Apache/nginx docroot
|    └── static/              # labs-striker-staticfiles submodule
├── requirements.txt          # `pip --freeze` generated requirements
├── scap/
|    ├── checks/
|    |    └──  virtualenv.sh  # Check script to rebuild venv on target hosts
|    ├── checks.yaml          # Post-deploy checks for Scap3
|    ├── dsh/                  # Scap3 target host sets
|    ├── log/                  # Generated by scap3, only on deploy master
|    └── scap.cfg              # Scap3 config file
├── striker/                  # striker submodule
|    └── requirements.txt      # Product level requirements
└── wheels/                    # labs-striker-wheels submodule
</pre>


== Submodules ==
== Submodules ==

Revision as of 04:32, 17 October 2017

The Striker build and deploy process involves updating the labs-striker-deploy git repository and then pushing the repository out to servers running Striker via scap3.

Deploy directory structure

/srv/deployment/striker/deploy/
 ├── _build/                    # Generated by make_wheels.sh, only on Cloud VPS staging server
 |    └── venv/                 # virtualenv used for creating wheels & top level 
 ├── ddl/                       # Manually created SQL patches
 ├── make_wheels.sh             # Build script for preparing labs-striker-wheels patches
 ├── public_html/               # Apache/nginx docroot
 |    └── static/               # labs-striker-staticfiles submodule
 ├── requirements.txt           # `pip --freeze` generated requirements
 ├── scap/
 |    ├── checks/
 |    |    └──  virtualenv.sh   # Check script to rebuild venv on target hosts
 |    ├── checks.yaml           # Post-deploy checks for Scap3
 |    ├── dsh/                  # Scap3 target host sets
 |    ├── log/                  # Generated by scap3, only on deploy master
 |    └── scap.cfg              # Scap3 config file
 ├── striker/                   # striker submodule
 |    └── requirements.txt      # Product level requirements
 └── wheels/                    # labs-striker-wheels submodule

Submodules

Most of the interesting code changes will happen in submodules of labs-striker-deploy:

labs-striker

There is nothing special to do to prepare this repo for deployment. Merge commits using Gerrit.

Changes made here however will trigger changes to other submodules. The the point a release is being made:

  • labs-striker-staticfiles must be updated if any changes were made to requirements.txt or if files were modified in the static directory.
  • labs-striker-wheels must be updated if any changes were made to requirements.txt.

labs-striker-staticfiles

These static assets are collected from the installed Django applications using Django's collectstatic management script. The contrib/collectstatic.sh convenience script is provided in labs-striker for collecting the files. This can be run in a MediaWiki-Vagrant development environment.

$ cd /vagrant/srv/striker
$ contrib/collectstatic.sh
Deleting 'robots.f71d20196d4c.txt'
Deleting 'robots.txt'
Deleting 'staticfiles.json'
...
Post-processed 'robots.txt' as 'robots.f71d20196d4c.txt'

110 static files copied to '/vagrant/srv/striker/staticfiles', 110 post-processed.
$ cd staticfiles
$ python -mjson.tool staticfiles.json > staticfiles.json.pretty
$ mv staticfiles.json.pretty staticfiles.json
$ cp -R . $PATH_TO_LAB_STRIKER_STATICFILES
$ cd $PATH_TO_LAB_STRIKER_STATICFILES
$ git status
$ git add .
$ git commit
$ git review

Before submitting you may want to look for old assets that can be removed as well. These would generally be <FILENAME>.<CONTENT_HASH>.<EXT> files that are for versions of the files that were removed from the manifest in a prior update that has already been deployed.

Testing in Cloud VPS project

TODO: explain how I typically stage pending changes in the striker Cloud VPS project.

Deploy

TODO: explain basic scap3 deployment