You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Deployments/Training

From Wikitech-static
< Deployments
Revision as of 00:47, 27 April 2021 by imported>Thcipriani
(diff) ← Older revision | Latest revision (diff) | Newer revision β†’ (diff)
Jump to navigation Jump to search

✨🌈Welcome, potential deployer!🌈✨

We're happy you're here! Whether you're new to deploying, or just need a refresher on the finer points of the the process for Wikimedia's production infrastructure: you've come to the right place. Our goal is to make deployments low-risk and a big component of that is incremental releases via backport.

The deployment system is never done: Write the doc you wish you had when you started, record a screencast, file a task for scap, find a relenger to rant at, make a patch β€” all encouraged!

But first, get training!

Get training

There are generally two Backport/config deployment windows per week which are designated as training slots; this means that the people who manage deployments for those specific windows have volunteered to help walk through the steps of deployment of any of the patches on the schedule. Look for "EU Backport and Config training" or "US Backport and Config training" in the Deployments schedule. There is also a Google meet scheduled during these windows, so that you can talk with the trainers live.

Sign up using the deployment training request form and be ready to take notes, to ssh into the deployment and mwdebug hosts, and to run commands.

You'll get to walk through one or more deployments of a backported MediaWiki change or a configuration change, starting with merging it, fetching it onto the deployment host, checking that it merges in cleanly and there's no uncommitted stuff around, putting it out to an mwdebug host for testing, and finally scapping it out to the entire cluster, all under the supervision of experienced deployers.

Don't be shy about asking questions. Have a question about how to manage a config change that includes multiple files? This is your opportunity to ask all about it, take notes, and help improve the docs about deployment!

If one training isn't enough, keep coming back. Do more deployments with someone more experienced around in case you run into "OMG how do I revert this?" and other such issues. That is what these training windows are for. Deploy, deploy, deploy!

When you've been through the training and you feel comfortable enough with deployments, you can help others by becoming a trainer. There's no official process to this; just sign up by adding your name as a deployer during the training window that's in the most convenient timezone, and don't forget to show up. Future deployers thank you in advance :-)

A summary of useful commands and other information is given below, and there is more detailed documentation available at Backport_windows/Deployers in case you prefer to read up in detail about everything ahead of time.

Useful information

Basic sync

  • All git activity happens in /srv/mediawiki-staging
  • +2 this patch
  • git fetch
  • git log -p HEAD..@{u}
  • scap sync-file [FILE] '[Message]'

Easy peasy.

Mw/Config change More advanced

Most of the time, you'll want to check the patch before it goes out. We do this on mwdebug servers.

  • +2 patch
  • git fetch
  • git log -p HEAD..@{u}
  • git rebase
  • [in another terminal] ssh mwdebug1002.eqiad.wmnet && scap pull
  • [check in browser]
  • scap sync-file [FILE] '[Message]'

MediaWiki patch

  • +2 patch for a deployed version of MediaWiki, skins, or extensions
  • cd /srv/medawiki-staging/php-[version]/ OR cd /srv/medawiki-staging/php-[version]/extension
  • git fetch
  • git log -p HEAD..@{u}
  • git rebase
  • [if extension] git submodule update
  • [in another window] ssh mwdebug1002.eqiad.wmnet && scap pull
  • [check in browser]
  • scap sync-file [FILE] '[Message]'

Helpful things to know

Deployment server directory structure
/srv/mediawiki-staging
β”œβ”€β”€ dblists
β”‚   β”œβ”€β”€ all.dblist
|   └── [...]
β”œβ”€β”€ images
β”œβ”€β”€ php -> php-1.36.0-wmf.26
β”œβ”€β”€ php-1.36.0-wmf.36
β”‚   β”œβ”€β”€ [...]
β”‚   β”œβ”€β”€ extensions
β”‚   β”‚   β”œβ”€β”€ 3D
β”‚   β”‚   β”œβ”€β”€ [...] 100s of extensions
β”‚   β”‚   └── ZeroPortal
β”‚   β”œβ”€β”€ skins
β”‚   β”‚   β”œβ”€β”€ CologneBlue
β”‚   β”‚   β”œβ”€β”€ [...] 8 skins
β”‚   β”‚   └── Vector
|   └── [...] # this is just mediawiki
β”œβ”€β”€ portals
β”œβ”€β”€ private
|   β”œβ”€β”€ [...]
β”‚   └── PrivateSettings.php
β”œβ”€β”€ README
β”œβ”€β”€ wikiversions.json
└── wmf-config
    β”œβ”€β”€ CommonSettings.php
    β”œβ”€β”€ [...]
    β”œβ”€β”€ InitialiseSettings.php
    └── [...]
Deployment server git structure
    • /srv/mediawiki-staging is a git repo.
    • /srv/mediawiki-staging/php-[version] is a git repo not a submodule
    • /srv/mediawiki-staging/php-[version]/extensions/[extension] is a submodule of php-[version]
    • /srv/mediawiki-staging/php-[version]/skins/[skin] is a submodule of php-[version]
    • These are the only files I've ever cared about as a deployer for many years:
/srv/mediawiki-staging
β”œβ”€β”€ .git
β”œβ”€β”€ php -> php-1.36.0-wmf.26
β”œβ”€β”€ php-1.36.0-wmf.36
β”‚   β”œβ”€β”€ .git
β”‚   β”œβ”€β”€ extensions
β”‚   β”‚   β”œβ”€β”€ 3D/.git
β”‚   β”‚   β”œβ”€β”€ [...] 100s submodules
β”‚   β”‚   └── ZeroPortal/.git
β”‚   β”œβ”€β”€ skins
β”‚   β”‚   β”œβ”€β”€ CologneBlue
β”‚   β”‚   β”œβ”€β”€ [...] 8 skins
β”‚   β”‚   └── Vector
|   └── [...] # this is just mediawiki
β”œβ”€β”€ portals
β”œβ”€β”€ private
|   β”œβ”€β”€ [...]
β”‚   └── PrivateSettings.php
β”œβ”€β”€ README
β”œβ”€β”€ wikiversions.json
└── wmf-config
    β”œβ”€β”€ CommonSettings.php
    β”œβ”€β”€ [...]
    β”œβ”€β”€ InitialiseSettings.php
    └── [...]