You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Server Admin Log: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

19 May 2022

  • curprev 00:5800:58, 19 May 2022imported>Stashbot 538,445 bytes +60,753 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance

18 May 2022

  • curprev 01:0501:05, 18 May 2022imported>Stashbot 477,692 bytes +34,747 ejegg: updated fundraising CiviCRM from d45afdfc to b8b8c177

16 May 2022

  • curprev 22:1422:14, 16 May 2022imported>Stashbot 442,945 bytes +16,328 jhathaway@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx2001.wikimedia.org with reason: exim debugging

15 May 2022

  • curprev 21:4721:47, 15 May 2022imported>Stashbot 426,617 bytes +1,183 aqu@deploy1002: Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 07s)

14 May 2022

  • curprev 08:3408:34, 14 May 2022imported>Stashbot 425,434 bytes +205 jynus@cumin1001: dbctl commit (dc=all): 'Depool db1172', diff saved to https://phabricator.wikimedia.org/P27830 and previous config saved to /var/cache/conftool/dbconfig/20220514-083421-jynus.json
  • curprev 00:5300:53, 14 May 2022imported>Stashbot 425,229 bytes +4,537 razzi@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on an-tool1005.eqiad.wmnet with reason: Server need to be downgraded to stretch, on monday

12 May 2022

  • curprev 21:5621:56, 12 May 2022imported>Stashbot 420,692 bytes +26,145 razzi@deploy1002: Finished deploy [analytics/turnilo/deploy@a2bdc3e]: (no justification provided) (duration: 02m 08s)

11 May 2022

  • curprev 22:2822:28, 11 May 2022imported>Stashbot 394,547 bytes +16,527 robh: cp305[67] returned to service and all green in icinga, cp305[89] depooling for firmware update T243167
  • curprev 01:4101:41, 11 May 2022imported>Stashbot 378,020 bytes +25,757 mutante: gitlab2001 - starting backup-restore service that had failed on previous automatic run

9 May 2022

  • curprev 21:5821:58, 9 May 2022imported>Stashbot 352,263 bytes +25,329 jhathaway@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx2001.wikimedia.org with reason: new kernel round deux

8 May 2022

  • curprev 07:1607:16, 8 May 2022imported>Stashbot 326,934 bytes +81 godog: silence probedown for thumbor:8800 until monday

7 May 2022

  • curprev 21:2921:29, 7 May 2022imported>Stashbot 326,853 bytes +2,312 andrew@deploy1002: Finished deploy [horizon/deploy@9d02cd6]: seeking consistency between codfw1dev and eqiad1 (duration: 04m 04s)

6 May 2022

  • curprev 19:1619:16, 6 May 2022imported>Stashbot 324,541 bytes +16,729 razzi@cumin1001: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-airflow1002.eqiad.wmnet
  • curprev 00:4600:46, 6 May 2022imported>Stashbot 307,812 bytes +83,868 rook@cumin1001: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host cloudvirt1016.eqiad.wmnet

5 May 2022

  • curprev 01:4201:42, 5 May 2022imported>Stashbot 223,944 bytes +94,859 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P27586 and previous config saved to /var/cache/conftool/dbconfig/20220505-014205-ladsgroup.json

4 May 2022

  • curprev 00:5000:50, 4 May 2022imported>Stashbot 129,085 bytes +36,153 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1100.eqiad.wmnet with reason: Maintenance

2 May 2022

  • curprev 23:1523:15, 2 May 2022imported>Stashbot 92,932 bytes +77,432 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host krb2002.codfw.wmnet with OS bullseye
  • curprev 00:5900:59, 2 May 2022imported>Stashbot 15,500 bytes +15,385 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P27203 and previous config saved to /var/cache/conftool/dbconfig/20220502-005940-ladsgroup.json

1 May 2022

29 April 2022

  • curprev 23:1123:11, 29 April 2022imported>Stashbot 1,095,940 bytes +87,748 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1149 (T306560)', diff saved to https://phabricator.wikimedia.org/P27163 and previous config saved to /var/cache/conftool/dbconfig/20220429-231136-ladsgroup.json
  • curprev 00:5700:57, 29 April 2022imported>Stashbot 1,008,192 bytes +87,341 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P26967 and previous config saved to /var/cache/conftool/dbconfig/20220429-005702-ladsgroup.json

28 April 2022

  • curprev 01:4701:47, 28 April 2022imported>Stashbot 920,851 bytes +89,525 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P26857 and previous config saved to /var/cache/conftool/dbconfig/20220428-014723-ladsgroup.json

27 April 2022

  • curprev 01:4301:43, 27 April 2022imported>Stashbot 831,326 bytes +86,215 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P26663 and previous config saved to /var/cache/conftool/dbconfig/20220427-014355-ladsgroup.json

25 April 2022

  • curprev 23:0523:05, 25 April 2022imported>Stashbot 745,111 bytes +49,942 mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mwdebug: apply
  • curprev 00:5400:54, 25 April 2022imported>Stashbot 695,169 bytes +25,636 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P26408 and previous config saved to /var/cache/conftool/dbconfig/20220425-005432-ladsgroup.json

24 April 2022

  • curprev 01:2801:28, 24 April 2022imported>Stashbot 669,533 bytes +28,655 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 12 hosts with reason: Maintenance

23 April 2022

  • curprev 01:3401:34, 23 April 2022imported>Stashbot 640,878 bytes +45,596 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T306560)', diff saved to https://phabricator.wikimedia.org/P26246 and previous config saved to /var/cache/conftool/dbconfig/20220423-013450-ladsgroup.json

22 April 2022

  • curprev 01:4701:47, 22 April 2022imported>Stashbot 595,282 bytes −356,134 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance

21 April 2022

  • curprev 00:5200:52, 21 April 2022imported>Stashbot 951,416 bytes +154,237 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P25837 and previous config saved to /var/cache/conftool/dbconfig/20220421-005225-ladsgroup.json

20 April 2022

  • curprev 01:3101:31, 20 April 2022imported>Stashbot 797,179 bytes +136,676 pt1979@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)

19 April 2022

  • curprev 00:5300:53, 19 April 2022imported>Stashbot 660,503 bytes +82,833 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P25214 and previous config saved to /var/cache/conftool/dbconfig/20220419-005334-ladsgroup.json

18 April 2022

  • curprev 01:4001:40, 18 April 2022imported>Stashbot 577,670 bytes +74,955 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1180', diff saved to https://phabricator.wikimedia.org/P24982 and previous config saved to /var/cache/conftool/dbconfig/20220418-014003-ladsgroup.json

17 April 2022

  • curprev 00:5100:51, 17 April 2022imported>Stashbot 502,715 bytes +28,006 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1147 (T298565)', diff saved to https://phabricator.wikimedia.org/P24761 and previous config saved to /var/cache/conftool/dbconfig/20220417-005150-ladsgroup.json

16 April 2022

  • curprev 00:3500:35, 16 April 2022imported>Stashbot 474,709 bytes +12,679 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298565)', diff saved to https://phabricator.wikimedia.org/P24681 and previous config saved to /var/cache/conftool/dbconfig/20220416-003538-ladsgroup.json

14 April 2022

  • curprev 22:2822:28, 14 April 2022imported>Stashbot 462,030 bytes +16,537 mutante: gitlab - deleting runner-1018, runner-1019, creating runner-1029, runner-1030 T297659
  • curprev 00:3700:37, 14 April 2022imported>Stashbot 445,493 bytes +50,258 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1140.eqiad.wmnet with reason: Maintenance

13 April 2022

  • curprev 01:4201:42, 13 April 2022imported>Stashbot 395,235 bytes +39,686 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1135', diff saved to https://phabricator.wikimedia.org/P24545 and previous config saved to /var/cache/conftool/dbconfig/20220413-014214-ladsgroup.json

12 April 2022

  • curprev 00:4900:49, 12 April 2022imported>Stashbot 355,549 bytes +52,260 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24477 and previous config saved to /var/cache/conftool/dbconfig/20220412-004933-ladsgroup.json

11 April 2022

  • curprev 01:4301:43, 11 April 2022imported>Stashbot 303,289 bytes +5,989 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24355 and previous config saved to /var/cache/conftool/dbconfig/20220411-014316-ladsgroup.json

9 April 2022

  • curprev 12:3912:39, 9 April 2022imported>Stashbot 297,300 bytes +1,710 godog: bounce prometheus@ops on prometheus5001
  • curprev 00:5300:53, 9 April 2022imported>Stashbot 295,590 bytes +31,885 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24333 and previous config saved to /var/cache/conftool/dbconfig/20220409-005351-ladsgroup.json

7 April 2022

  • curprev 22:1822:18, 7 April 2022imported>Stashbot 263,705 bytes +77,590 ejegg: restarted fundraising scheduled jobs
  • curprev 00:5800:58, 7 April 2022imported>Stashbot 186,115 bytes +64,418 marostegui@cumin1001: dbctl commit (dc=all): 'Depooling db1129 (T297189)', diff saved to https://phabricator.wikimedia.org/P24195 and previous config saved to /var/cache/conftool/dbconfig/20220407-005817-marostegui.json

6 April 2022

  • curprev 01:3401:34, 6 April 2022imported>Stashbot 121,697 bytes +61,769 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1105:3311', diff saved to https://phabricator.wikimedia.org/P24142 and previous config saved to /var/cache/conftool/dbconfig/20220406-013420-ladsgroup.json

5 April 2022

  • curprev 00:5800:58, 5 April 2022imported>Stashbot 59,928 bytes +52,310 sukhe@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp4034.ulsfo.wmnet

2 April 2022

  • curprev 11:2611:26, 2 April 2022imported>Stashbot 7,618 bytes +272 akosiaris: disable zotero paging until T291707 is resolved.

1 April 2022

  • curprev 23:2523:25, 1 April 2022imported>Stashbot 7,346 bytes −1,236,074 mutante: DNS - new project language 'kcg'. 'Tyap is a regionally important dialect cluster of Plateau languages in Nigeria's Middle Belt, named after its prestige dialect. It is also known by its Hausa exonym as Katab or Kataf.' T305279

31 March 2022

  • curprev 23:4523:45, 31 March 2022imported>Stashbot 1,243,420 bytes +66,130 mutante: gitlab2001 - fdisk /dev/vdb (g, w) (create partition table), (n, w) (create partition) ; mkfs.ext4 /dev/vdb1 (create filesystem); systemctl reset-failed (fix Icinga alert); mkdir /mnt/gitlab-backup; mount /dev/vdb1 /mnt/gitlab-backup ; blkid (get UUID); edit /etc/fstab and insert "UUID=c5235682-ac21-46a9-85ee-9603f694a6a4 /mnt/gitlab-backup ext4 errors=remount-ro 0 2" T274463
  • curprev 01:4401:44, 31 March 2022imported>Stashbot 1,177,290 bytes +118,543 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P23948 and previous config saved to /var/cache/conftool/dbconfig/20220331-014403-ladsgroup.json

30 March 2022

  • curprev 01:4601:46, 30 March 2022imported>Stashbot 1,058,747 bytes +98,930 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1146:3312 (T298565)', diff saved to https://phabricator.wikimedia.org/P23664 and previous config saved to /var/cache/conftool/dbconfig/20220330-014621-ladsgroup.json
(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)