You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
There are a lot of moving parts in our production stack -- MediaWiki, its extensions, various back-end services, HTTPS handlers, caches, just to name a few. It is thus important that you communicate your deployment schedules on the Deployments page.
The deployment process starts with updating the deploy repository. Go into your source repository and update it with:
$ ./server.js build --deploy-repo --force --review
The build script will update the pointer of the deploy repository's submodule, create a Docker container in which it will install the module dependencies and send the changes to Gerrit. Review them and merge. Next, log onto deployment.eqiad.wmnet and update the repo there:
$ cd /srv/deployment/<service-name>/deploy $ git pull && git submodule update --init
In the #wikimedia-operations IRC channel announce the deployment by logging it into the Server Admin Log with
!log <service-name> deploying <deploy-repo-sha1>. Now, proceed to do the deployment from deployment.eqiad.wmnet:
Scap3 will deploy the code, restart the service and check its port and health. In case it detects some problems on the canary node, it will suggest to perform a roll-back. Otherwise it will proceed to deploying it to the rest of the nodes, which completes the deployment process.
Dealing with Problems
Scap3 includes a utility which can be used to monitor the output of the commands executed on the target nodes. Fire up a second terminal, connect to deployment.eqiad.wmnet and execute the
deploy-log command from /srv/deployment/<service-name>/deploy before starting the deployment. The output should help you figure out what went wrong.
If you haven't started an instance of deploy-log during the deploy, but it went badly, you can still recuperate the logs by running
Reverting a Deployment
Sometimes the deployment process goes well, but the code that was deployed isn't functioning properly. To revert a deployment and bring the code on the target nodes to a previous state, find the deploy repository's SHA1 that contained the good code and then deploy it with:
$ deploy --rev <sha1>