You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Server Admin Log: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)

1 July 2022

  • curprev 01:3901:39, 1 July 2022imported>Stashbot 3,918 bytes −776,405 krinkle@deploy1002: Synchronized tests/: I60edfb0f60 (1/3) (duration: 03m 32s)

30 June 2022

  • curprev 01:3601:36, 30 June 2022imported>Stashbot 780,323 bytes +52,007 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2158.codfw.wmnet with OS bullseye

29 June 2022

  • curprev 00:1800:18, 29 June 2022imported>Stashbot 728,316 bytes +67,167 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudgw2003-dev.codfw.wmnet with OS bullseye

27 June 2022

  • curprev 23:5123:51, 27 June 2022imported>Stashbot 661,149 bytes +85,388 cmjohnson@cumin1001: START - Cookbook sre.hosts.provision for host mw1474.mgmt.eqiad.wmnet with reboot policy FORCED
  • curprev 01:2501:25, 27 June 2022imported>Stashbot 575,761 bytes +7,333 cmjohnson@cumin1001: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-worker1008.mgmt.eqiad.wmnet with reboot policy FORCED

25 June 2022

  • curprev 18:1718:17, 25 June 2022imported>Stashbot 568,428 bytes +2,028 jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest1002.eqiad.wmnet

24 June 2022

  • curprev 19:3519:35, 24 June 2022imported>Stashbot 566,400 bytes +54,579 dancy@deploy1002: backport aborted: (duration: 00m 12s)

23 June 2022

  • curprev 21:2321:23, 23 June 2022imported>Stashbot 511,821 bytes +39,618 mutante: restbase-dev1006 has manually installed packages (wrk, maybe others)
  • curprev 00:3500:35, 23 June 2022imported>Stashbot 472,203 bytes +29,515 brennen: end of phabricator maintenance window

22 June 2022

  • curprev 01:1801:18, 22 June 2022imported>Stashbot 442,688 bytes +15,474 mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mwdebug: apply

20 June 2022

  • curprev 07:1407:14, 20 June 2022imported>Stashbot 427,214 bytes +308 SandraEbele: Started Airflow 3 Wikidata metrics jobs (Articleplaceholder, Reliability and SpecialEntityData metrics).

19 June 2022

  • curprev 10:2810:28, 19 June 2022imported>Stashbot 426,906 bytes +493 ayounsi@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1132.eqiad.wmnet with reason: depooled

17 June 2022

  • curprev 22:0522:05, 17 June 2022imported>Stashbot 426,413 bytes +16,273 AndyRussG: update payments-wiki revision 10304f69 -> ef53c82e
  • curprev 01:4301:43, 17 June 2022imported>Stashbot 410,140 bytes +29,970 pt1979@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1017.eqiad.wmnet with reason: host reimage

15 June 2022

  • curprev 22:4822:48, 15 June 2022imported>Stashbot 380,170 bytes +61,049 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1184 (T310011)', diff saved to https://phabricator.wikimedia.org/P29867 and previous config saved to /var/cache/conftool/dbconfig/20220615-224845-marostegui.json

14 June 2022

  • curprev 23:5223:52, 14 June 2022imported>Stashbot 319,121 bytes +44,141 mutante: gitlab-runner1001/1002 - clean revert not possible, icinga alerting about failed buildkitd service, manually deleting systemd unit and trying to clean up T308271
  • curprev 00:3600:36, 14 June 2022imported>Stashbot 274,980 bytes +45,898 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1147 (T310011)', diff saved to https://phabricator.wikimedia.org/P29701 and previous config saved to /var/cache/conftool/dbconfig/20220614-003608-marostegui.json

12 June 2022

  • curprev 18:3118:31, 12 June 2022imported>Stashbot 229,082 bytes +4,306 andrew@cumin1001: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddumps1002.wikimedia.org with OS bullseye
  • curprev 01:4601:46, 12 June 2022imported>Stashbot 224,776 bytes +4,304 andrew@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage

11 June 2022

  • curprev 01:1701:17, 11 June 2022imported>Stashbot 220,472 bytes +8,628 mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mwdebug: apply

10 June 2022

  • curprev 00:3300:33, 10 June 2022imported>Stashbot 211,844 bytes +35,139 ejegg: rolled back payments-wiki from 05139a0c to 8c6208c2

9 June 2022

  • curprev 00:4900:49, 9 June 2022imported>Stashbot 176,705 bytes +52,552 krinkle@deploy1002: Synchronized php-1.39.0-wmf.15/includes/libs/rdbms/: I99b817b3d50ffcdf56, T310214 (duration: 03m 23s)

8 June 2022

  • curprev 01:4301:43, 8 June 2022imported>Stashbot 124,153 bytes +33,565 cstone: civicrm revision changed from de12571a to b0b400ae

6 June 2022

  • curprev 23:1723:17, 6 June 2022imported>Stashbot 90,588 bytes +16,595 tzatziki: removing one file for legal compliance

5 June 2022

  • curprev 22:2122:21, 5 June 2022imported>Stashbot 73,993 bytes +6,438 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1144:3314 (T298560)', diff saved to https://phabricator.wikimedia.org/P29417 and previous config saved to /var/cache/conftool/dbconfig/20220605-222110-ladsgroup.json
  • curprev 01:3701:37, 5 June 2022imported>Stashbot 67,555 bytes +6,227 andrew@cumin1001: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye

3 June 2022

  • curprev 22:1922:19, 3 June 2022imported>Stashbot 61,328 bytes +9,538 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance
  • curprev 01:2001:20, 3 June 2022imported>Stashbot 51,790 bytes +28,593 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1121 (T298560)', diff saved to https://phabricator.wikimedia.org/P29365 and previous config saved to /var/cache/conftool/dbconfig/20220603-012045-ladsgroup.json

2 June 2022

  • curprev 01:4701:47, 2 June 2022imported>Stashbot 23,197 bytes −1,118,222 mwdebug-deploy@deploy1002: helmfile [eqiad] START helmfile.d/services/mwdebug: apply

1 June 2022

  • curprev 01:4101:41, 1 June 2022imported>Stashbot 1,141,419 bytes +62,344 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance

31 May 2022

  • curprev 00:4000:40, 31 May 2022imported>Stashbot 1,079,075 bytes +101,499 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance

30 May 2022

  • curprev 01:4501:45, 30 May 2022imported>Stashbot 977,576 bytes +7,857 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1113:3316', diff saved to https://phabricator.wikimedia.org/P28904 and previous config saved to /var/cache/conftool/dbconfig/20220530-014458-ladsgroup.json

28 May 2022

  • curprev 23:3623:36, 28 May 2022imported>Stashbot 969,719 bytes +50,883 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1101:3317 (T298560)', diff saved to https://phabricator.wikimedia.org/P28882 and previous config saved to /var/cache/conftool/dbconfig/20220528-233650-ladsgroup.json
  • curprev 01:3201:32, 28 May 2022imported>Stashbot 918,836 bytes +45,130 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1122 (T309311)', diff saved to https://phabricator.wikimedia.org/P28737 and previous config saved to /var/cache/conftool/dbconfig/20220528-013212-ladsgroup.json

27 May 2022

  • curprev 00:4500:45, 27 May 2022imported>Stashbot 873,706 bytes +31,398 mutante: rsyncing /srv/gitlab-backup from gitlab1004 to gitlab2002 | systemctl status full-backup ..in progress on gitlab1001 - T274463

26 May 2022

  • curprev 00:5800:58, 26 May 2022imported>Stashbot 842,308 bytes +49,509 mutante: gitlab1001 - T308089 T274463 - gitlab1001 - systemctl start full-backup

25 May 2022

  • curprev 00:1500:15, 25 May 2022imported>Stashbot 792,799 bytes +52,401 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1096:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28462 and previous config saved to /var/cache/conftool/dbconfig/20220525-001552-ladsgroup.json

24 May 2022

  • curprev 00:5200:52, 24 May 2022imported>Stashbot 740,398 bytes +67,605 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P28379 and previous config saved to /var/cache/conftool/dbconfig/20220524-005257-ladsgroup.json

22 May 2022

  • curprev 20:4620:46, 22 May 2022imported>Stashbot 672,793 bytes +13,528 mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mwdebug: apply
  • curprev 00:2100:21, 22 May 2022imported>Stashbot 659,265 bytes +20,709 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1146:3312 (T298560)', diff saved to https://phabricator.wikimedia.org/P28249 and previous config saved to /var/cache/conftool/dbconfig/20220522-002120-ladsgroup.json

21 May 2022

  • curprev 01:0601:06, 21 May 2022imported>Stashbot 638,556 bytes +27,942 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1158 (T298555)', diff saved to https://phabricator.wikimedia.org/P28208 and previous config saved to /var/cache/conftool/dbconfig/20220521-010640-ladsgroup.json

20 May 2022

  • curprev 01:3101:31, 20 May 2022imported>Stashbot 610,614 bytes +72,169 cmooney@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)

19 May 2022

  • curprev 00:5800:58, 19 May 2022imported>Stashbot 538,445 bytes +60,753 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance

18 May 2022

  • curprev 01:0501:05, 18 May 2022imported>Stashbot 477,692 bytes +34,747 ejegg: updated fundraising CiviCRM from d45afdfc to b8b8c177

16 May 2022

  • curprev 22:1422:14, 16 May 2022imported>Stashbot 442,945 bytes +16,328 jhathaway@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx2001.wikimedia.org with reason: exim debugging

15 May 2022

  • curprev 21:4721:47, 15 May 2022imported>Stashbot 426,617 bytes +1,183 aqu@deploy1002: Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 07s)

14 May 2022

  • curprev 08:3408:34, 14 May 2022imported>Stashbot 425,434 bytes +205 jynus@cumin1001: dbctl commit (dc=all): 'Depool db1172', diff saved to https://phabricator.wikimedia.org/P27830 and previous config saved to /var/cache/conftool/dbconfig/20220514-083421-jynus.json
  • curprev 00:5300:53, 14 May 2022imported>Stashbot 425,229 bytes +4,537 razzi@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on an-tool1005.eqiad.wmnet with reason: Server need to be downgraded to stretch, on monday

12 May 2022

  • curprev 21:5621:56, 12 May 2022imported>Stashbot 420,692 bytes +26,145 razzi@deploy1002: Finished deploy [analytics/turnilo/deploy@a2bdc3e]: (no justification provided) (duration: 02m 08s)

11 May 2022

  • curprev 22:2822:28, 11 May 2022imported>Stashbot 394,547 bytes +16,527 robh: cp305[67] returned to service and all green in icinga, cp305[89] depooling for firmware update T243167
(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)