You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Release Engineering/Drafts/Deployments/How to pair on the train

From Wikitech-static
Jump to navigation Jump to search

Background

Until October of 2019, running the deployment train was done by one assigned person in two-week shifts. In an effort to better share the workload and improve continuity of knowledge between successive deployers, Release Engineering then moved to a staggered pairing rotation. Each train has a primary conductor and a backup conductor. The following week, the primary moves to backup and a new primary is assigned.

Helpful practices

On Monday, begin by communicating with your partner and establishing how you'll collaborate over the course of the week.

If you are going to be away from your computer during or soon after deploy windows, let your partner know and coordinate coverage for those times. For example, if you have an appointment or family commitments after the deploy on Wednesday, ask your backup conductor to watch logs and the blocker task during that time.

These are rough guidelines for sharing the work, and should be improved as we learn more.

  • Updates on IRC while your partner is working and updates on the train blocker ticket if they're offline seems to be a useful pattern.
  • In general, communicate decisions on the blocker ticket and make use of the Server Admin Log.
  • Liberal use of video chat for pairing on hard problems is encouraged.
  • It works well to have the primary do the work of cutting the branch, syncing wikis, etc., while the backup keeps an eye on logs, works on improvements to deploy tooling, and is generally an extra pair of eyes for the whole process.
  • If you are in doubt about anything and it's during your partner's working hours, consult them first and get their help in resolving your questions.
  • If one member of the pair is in the European window and one is in the American window, both train deployment windows should be reserved on the Deployments calendar. This gives a backup deployer a defined window for moving the train forward outside the primary's working hours, if it becomes necessary.
  • If the train is blocked or there are any other issues, communicate the transfer of responsibility on the train blocker ticket by assigning it to the responsible party and leaving a note.