You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Deployments/Training"

From Wikitech-static
Jump to navigation Jump to search
imported>ArielGlenn
(Add a section on getting trained: how to sign up, what preparation you need, what to expect.)
imported>Thcipriani
Β 
Line 1: Line 1:
{{Note|content=Find deployment training on the Google Deployment Calendar <code>wikimedia.org_rudis09ii2mm5fk4hgdjeh1u64@group.calendar.google.com</code>}}
{{Note|content=Find deployment training on the Google Deployment Calendar <code>wikimedia.org_rudis09ii2mm5fk4hgdjeh1u64@group.calendar.google.com</code>}}


Welcome, potential deployer!
[[Image:I-bork-the-wikis-merit-badge.png|400px|frameless|center|middle]]
<templatestyles src="Template:FancyDoc/style.css" />
<div class="fancydoc-header fancydoc--green-border">
'''✨🌈Welcome, potential deployer!🌈✨'''


Deployments cause anxiety because there is a potential to break Wikipedia which is a site that billions of people use{{cn}}. The deployment system has evolved defense mechanisms that make Wikipedia harder to break. You do not need to be a deployment tooling expert to deploy Wikipedia; however, to reduce the anxiety of deployment you need to deploy.
We're happy you're here! Whether you're new to deploying, or just need a refresher on the finer points of the the process for Wikimedia's production infrastructure: you've come to the right place. Our goal is to make [https://www.informit.com/articles/article.aspx?p=1833567 deployments low-risk] and a big component of that is incremental releases via backport. Β 
Β 
The old saw, "[https://www.martinfowler.com/bliki/FrequencyReducesDifficulty.html if it hurts, do it more]" applies to deploying your code.


The deployment system is never done: Write the doc you wish you had when you started, record a screencast, file a task for <code>scap</code>, find a relenger to rant at, make a patch β€” all encouraged!
The deployment system is never done: Write the doc you wish you had when you started, record a screencast, file a task for <code>scap</code>, find a relenger to rant at, make a patch β€” all encouraged!


But first, get trained!
But first, get training!
</div>


== Getting trained ==
<span id="getting_trained"></span><span id="Getting_trained"></span>
== Get training ==


There are generally two Backport/config deployment windows per week which are designated as training slots; this means that the people who manage deployments for those specific windows have volunteered to help walk through the steps of deployment of any of the patches on the schedule. Look for "EU Backport and Config training" or "US Backport and Config training" in the [[Deployments]] schedule.Β  There is also a Google meet scheduled during these windows, so that you can talk with the trainers live.
There are generally two Backport/config deployment windows per week which are designated as training slots; this means that the people who manage deployments for those specific windows have volunteered to help walk through the steps of deployment of any of the patches on the schedule. Look for "EU Backport and Config training" or "US Backport and Config training" in the [[Deployments]] schedule.Β  There is also a Google meet scheduled during these windows, so that you can talk with the trainers live.


You don't have to sign up anywhere in advance. Just show up in the #wikimedia-operations IRC channel on freenode and be ready to take notes, to ssh into the deployment and mwdebug hosts, and to run commands.
Sign up using the [https://phabricator.wikimedia.org/maniphest/task/edit/form/96/ deployment training request form] and be ready to take notes, to ssh into the deployment and mwdebug hosts, and to run commands.


You'll get to walk through one or more deployments of a backported MediaWiki change or a configuration change, starting with merging it, fetching it onto the deployment host, checking that it merges in cleanly and there's no uncommitted stuff around, putting it out to an mwdebug host for testing, and finally scapping it out to the entire cluster, all under the supervision of experienced deployers.
You'll get to walk through one or more deployments of a backported MediaWiki change or a configuration change, starting with merging it, fetching it onto the deployment host, checking that it merges in cleanly and there's no uncommitted stuff around, putting it out to an mwdebug host for testing, and finally scapping it out to the entire cluster, all under the supervision of experienced deployers.

Latest revision as of 00:47, 27 April 2021

✨🌈Welcome, potential deployer!🌈✨

We're happy you're here! Whether you're new to deploying, or just need a refresher on the finer points of the the process for Wikimedia's production infrastructure: you've come to the right place. Our goal is to make deployments low-risk and a big component of that is incremental releases via backport.

The deployment system is never done: Write the doc you wish you had when you started, record a screencast, file a task for scap, find a relenger to rant at, make a patch β€” all encouraged!

But first, get training!

Get training

There are generally two Backport/config deployment windows per week which are designated as training slots; this means that the people who manage deployments for those specific windows have volunteered to help walk through the steps of deployment of any of the patches on the schedule. Look for "EU Backport and Config training" or "US Backport and Config training" in the Deployments schedule. There is also a Google meet scheduled during these windows, so that you can talk with the trainers live.

Sign up using the deployment training request form and be ready to take notes, to ssh into the deployment and mwdebug hosts, and to run commands.

You'll get to walk through one or more deployments of a backported MediaWiki change or a configuration change, starting with merging it, fetching it onto the deployment host, checking that it merges in cleanly and there's no uncommitted stuff around, putting it out to an mwdebug host for testing, and finally scapping it out to the entire cluster, all under the supervision of experienced deployers.

Don't be shy about asking questions. Have a question about how to manage a config change that includes multiple files? This is your opportunity to ask all about it, take notes, and help improve the docs about deployment!

If one training isn't enough, keep coming back. Do more deployments with someone more experienced around in case you run into "OMG how do I revert this?" and other such issues. That is what these training windows are for. Deploy, deploy, deploy!

When you've been through the training and you feel comfortable enough with deployments, you can help others by becoming a trainer. There's no official process to this; just sign up by adding your name as a deployer during the training window that's in the most convenient timezone, and don't forget to show up. Future deployers thank you in advance :-)

A summary of useful commands and other information is given below, and there is more detailed documentation available at Backport_windows/Deployers in case you prefer to read up in detail about everything ahead of time.

Useful information

Basic sync

  • All git activity happens in /srv/mediawiki-staging
  • +2 this patch
  • git fetch
  • git log -p HEAD..@{u}
  • scap sync-file [FILE] '[Message]'

Easy peasy.

Mw/Config change More advanced

Most of the time, you'll want to check the patch before it goes out. We do this on mwdebug servers.

  • +2 patch
  • git fetch
  • git log -p HEAD..@{u}
  • git rebase
  • [in another terminal] ssh mwdebug1002.eqiad.wmnet && scap pull
  • [check in browser]
  • scap sync-file [FILE] '[Message]'

MediaWiki patch

  • +2 patch for a deployed version of MediaWiki, skins, or extensions
  • cd /srv/medawiki-staging/php-[version]/ OR cd /srv/medawiki-staging/php-[version]/extension
  • git fetch
  • git log -p HEAD..@{u}
  • git rebase
  • [if extension] git submodule update
  • [in another window] ssh mwdebug1002.eqiad.wmnet && scap pull
  • [check in browser]
  • scap sync-file [FILE] '[Message]'

Helpful things to know

Deployment server directory structure
/srv/mediawiki-staging
β”œβ”€β”€ dblists
β”‚   β”œβ”€β”€ all.dblist
|   └── [...]
β”œβ”€β”€ images
β”œβ”€β”€ php -> php-1.36.0-wmf.26
β”œβ”€β”€ php-1.36.0-wmf.36
β”‚   β”œβ”€β”€ [...]
β”‚   β”œβ”€β”€ extensions
β”‚   β”‚   β”œβ”€β”€ 3D
β”‚   β”‚   β”œβ”€β”€ [...] 100s of extensions
β”‚   β”‚   └── ZeroPortal
β”‚   β”œβ”€β”€ skins
β”‚   β”‚   β”œβ”€β”€ CologneBlue
β”‚   β”‚   β”œβ”€β”€ [...] 8 skins
β”‚   β”‚   └── Vector
|   └── [...] # this is just mediawiki
β”œβ”€β”€ portals
β”œβ”€β”€ private
|   β”œβ”€β”€ [...]
β”‚   └── PrivateSettings.php
β”œβ”€β”€ README
β”œβ”€β”€ wikiversions.json
└── wmf-config
    β”œβ”€β”€ CommonSettings.php
    β”œβ”€β”€ [...]
    β”œβ”€β”€ InitialiseSettings.php
    └── [...]
Deployment server git structure
    • /srv/mediawiki-staging is a git repo.
    • /srv/mediawiki-staging/php-[version] is a git repo not a submodule
    • /srv/mediawiki-staging/php-[version]/extensions/[extension] is a submodule of php-[version]
    • /srv/mediawiki-staging/php-[version]/skins/[skin] is a submodule of php-[version]
    • These are the only files I've ever cared about as a deployer for many years:
/srv/mediawiki-staging
β”œβ”€β”€ .git
β”œβ”€β”€ php -> php-1.36.0-wmf.26
β”œβ”€β”€ php-1.36.0-wmf.36
β”‚   β”œβ”€β”€ .git
β”‚   β”œβ”€β”€ extensions
β”‚   β”‚   β”œβ”€β”€ 3D/.git
β”‚   β”‚   β”œβ”€β”€ [...] 100s submodules
β”‚   β”‚   └── ZeroPortal/.git
β”‚   β”œβ”€β”€ skins
β”‚   β”‚   β”œβ”€β”€ CologneBlue
β”‚   β”‚   β”œβ”€β”€ [...] 8 skins
β”‚   β”‚   └── Vector
|   └── [...] # this is just mediawiki
β”œβ”€β”€ portals
β”œβ”€β”€ private
|   β”œβ”€β”€ [...]
β”‚   └── PrivateSettings.php
β”œβ”€β”€ README
β”œβ”€β”€ wikiversions.json
└── wmf-config
    β”œβ”€β”€ CommonSettings.php
    β”œβ”€β”€ [...]
    β”œβ”€β”€ InitialiseSettings.php
    └── [...]