You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Services/Deployment"

From Wikitech-static
Jump to navigation Jump to search
imported>Mobrovac
m
 
imported>BryanDavis
(spelling)
Line 9: Line 9:
</pre>
</pre>


The build script will update the pointer of the deploy repository's submodule, create a Docker container in which it will install the module dependencies and send the changes to Gerrit. Review them and merge. Next, log onto `deployment.eqiad.wmnet` and update the repo there:
The build script will update the pointer of the deploy repository's submodule, create a Docker container in which it will install the module dependencies and send the changes to Gerrit. Review them and merge. Next, log onto [[deployment.eqiad.wmnet]] and update the repo there:


<pre>
<pre>
Line 16: Line 16:
</pre>
</pre>


In the [irc://irc.freenode.net/wikimedia-operations <tt>#wikimedia-operations</tt>] IRC channel announce the deployment by logging it into the [[Server Admin Log]] with <code>!log <service-name> deploying <deploy-repo-sha1></code>. Now, proceed to do the dpeloyment from <tt>deployment.eqiad.wmnet</tt>:
In the [irc://irc.freenode.net/wikimedia-operations <tt>#wikimedia-operations</tt>] IRC channel announce the deployment by logging it into the [[Server Admin Log]] with <code>!log <service-name> deploying <deploy-repo-sha1></code>. Now, proceed to do the deployment from <tt>deployment.eqiad.wmnet</tt>:


<pre>
<pre>

Revision as of 05:16, 22 April 2016

Regular Deployment

There are a lot of moving parts in our production stack -- MediaWiki, its extensions, various back-end services, HTTPS handlers, caches, just to name a few. It is thus important that you communicate your deployment schedules on the Deployments page.

The deployment process starts with updating the deploy repository. Go into your source repository and update it with:

$ ./server.js build --deploy-repo --force --review

The build script will update the pointer of the deploy repository's submodule, create a Docker container in which it will install the module dependencies and send the changes to Gerrit. Review them and merge. Next, log onto deployment.eqiad.wmnet and update the repo there:

$ cd /srv/deployment/<service-name>/deploy
$ git pull && git submodule update --init

In the #wikimedia-operations IRC channel announce the deployment by logging it into the Server Admin Log with !log <service-name> deploying <deploy-repo-sha1>. Now, proceed to do the deployment from deployment.eqiad.wmnet:

$ deploy

Scap3 will deploy the code, restart the service and check its port and health. In case it detects some problems on the canary node, it will suggest to perform a roll-back. Otherwise it will proceed to deploying it to the rest of the nodes, which completes the deployment process.

Dealing with Problems

Deployment Debugging

Scap3 includes a utility which can be used to monitor the output of the commands executed on the target nodes. Fire up a second terminal, connect to deployment.eqiad.wmnet and execute the deploy-log command from /srv/deployment/<service-name>/deploy before starting the deployment. The output should help you figure out what went wrong.

If you haven't started an instance of deploy-log during the deploy, but it went badly, you can still recuperate the logs by running deploy-log --latest.

Reverting a Deployment

Sometimes the deployment process goes well, but the code that was deployed isn't functioning properly. To revert a deployment and bring the code on the target nodes to a previous state, find the deploy repository's SHA1 that contained the good code and then deploy it with:

$ deploy --rev <sha1>