You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Server Admin Log: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)

9 December 2022

  • curprev 01:1101:11, 9 December 2022imported>Stashbot 384,656 bytes +64,721 eevans@cumin1001: START - Cookbook sre.hosts.reimage for host cassandra-dev2002.codfw.wmnet with OS buster

8 December 2022

  • curprev 01:0501:05, 8 December 2022imported>Stashbot 319,935 bytes +71,471 bblack: lvsNNNN: restart pybal to apply etcd key changes on all "high-traffic1" lvs at all sites - T324336

7 December 2022

  • curprev 00:2100:21, 7 December 2022imported>Stashbot 248,464 bytes +42,999 cmjohnson@cumin1001: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host kubernetes1024.eqiad.wmnet with OS bullseye

6 December 2022

  • curprev 01:2501:25, 6 December 2022imported>Stashbot 205,465 bytes +81,524 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P42379 and previous config saved to /var/cache/conftool/dbconfig/20221206-012539-ladsgroup.json

4 December 2022

  • curprev 04:1904:19, 4 December 2022imported>Stashbot 123,941 bytes +177 TheresNoTime: T302486 : `[samtar@mwmaint1002 ~]$ mwscript maintenance/fixMergeHistoryCorruption.php --wiki enwiki --dry-run --ns 828`

3 December 2022

  • curprev 00:1700:17, 3 December 2022imported>Stashbot 123,764 bytes +12,040 cwhite: draining shards from logstash1010, logstash1033, logstash1034, logstash1035 - T321410

2 December 2022

  • curprev 00:0900:09, 2 December 2022imported>Stashbot 111,724 bytes −1,825,217 rzl@cumin1001: conftool action : set/pooled=no; selector: name=mw14(45|46).eqiad.wmnet,cluster=jobrunner

30 November 2022

  • curprev 01:2201:22, 30 November 2022imported>Stashbot 1,936,941 bytes +129,140 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2176 (T321126)', diff saved to https://phabricator.wikimedia.org/P41834 and previous config saved to /var/cache/conftool/dbconfig/20221130-012218-marostegui.json

29 November 2022

  • curprev 01:1701:17, 29 November 2022imported>Stashbot 1,807,801 bytes +121,792 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1166 (T322618)', diff saved to https://phabricator.wikimedia.org/P41530 and previous config saved to /var/cache/conftool/dbconfig/20221129-011707-ladsgroup.json

27 November 2022

  • curprev 03:0103:01, 27 November 2022imported>Stashbot 1,686,009 bytes +943 ladsgroup@cumin1001: dbctl commit (dc=all): 'db2105 (re)pooling @ 100%: Maint', diff saved to https://phabricator.wikimedia.org/P41257 and previous config saved to /var/cache/conftool/dbconfig/20221127-030126-ladsgroup.json

26 November 2022

  • curprev 21:3421:34, 26 November 2022imported>Stashbot 1,685,066 bytes +3,790 urandom: initiating Cassandra bootstrap, aqs1021-b -- T307802
  • curprev 01:1601:16, 26 November 2022imported>Stashbot 1,681,276 bytes +58,470 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P41244 and previous config saved to /var/cache/conftool/dbconfig/20221126-011647-ladsgroup.json

25 November 2022

  • curprev 01:1801:18, 25 November 2022imported>Stashbot 1,622,806 bytes +76,429 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P41071 and previous config saved to /var/cache/conftool/dbconfig/20221125-011818-marostegui.json

24 November 2022

  • curprev 01:2601:26, 24 November 2022imported>Stashbot 1,546,377 bytes +91,450 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2169:3317 (T321126)', diff saved to https://phabricator.wikimedia.org/P40894 and previous config saved to /var/cache/conftool/dbconfig/20221124-012652-marostegui.json

23 November 2022

  • curprev 01:1601:16, 23 November 2022imported>Stashbot 1,454,927 bytes +128,094 sukhe@cumin2002: START - Cookbook sre.hosts.reimage for host cp2041.codfw.wmnet with OS bullseye

22 November 2022

  • curprev 01:1401:14, 22 November 2022imported>Stashbot 1,326,833 bytes +77,259 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T323214)', diff saved to https://phabricator.wikimedia.org/P40411 and previous config saved to /var/cache/conftool/dbconfig/20221122-011404-ladsgroup.json

21 November 2022

  • curprev 01:0801:08, 21 November 2022imported>Stashbot 1,249,574 bytes +1,268 sukhe@cumin2002: START - Cookbook sre.hosts.reimage for host cp5029.eqsin.wmnet with OS buster

19 November 2022

  • curprev 22:5122:51, 19 November 2022imported>Stashbot 1,248,306 bytes +1,417 sukhe@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5020.eqsin.wmnet with OS buster
  • curprev 01:2401:24, 19 November 2022imported>Stashbot 1,246,889 bytes +59,627 sukhe@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5018.eqsin.wmnet with reason: host reimage

18 November 2022

  • curprev 01:2601:26, 18 November 2022imported>Stashbot 1,187,262 bytes +56,176 pt1979@cumin2002: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['db2173']

17 November 2022

  • curprev 00:5900:59, 17 November 2022imported>Stashbot 1,131,086 bytes +67,747 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db2125 (T323214)', diff saved to https://phabricator.wikimedia.org/P40031 and previous config saved to /var/cache/conftool/dbconfig/20221117-005929-ladsgroup.json

16 November 2022

  • curprev 01:1501:15, 16 November 2022imported>Stashbot 1,063,339 bytes +102,599 sukhe@cumin2002: START - Cookbook sre.hosts.reimage for host cp2042.codfw.wmnet with OS bullseye

15 November 2022

  • curprev 01:0701:07, 15 November 2022imported>Stashbot 960,740 bytes +99,923 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2136', diff saved to https://phabricator.wikimedia.org/P39630 and previous config saved to /var/cache/conftool/dbconfig/20221115-010745-marostegui.json

12 November 2022

  • curprev 23:3423:34, 12 November 2022imported>Stashbot 860,817 bytes +12,581 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2179 (T318605)', diff saved to https://phabricator.wikimedia.org/P39371 and previous config saved to /var/cache/conftool/dbconfig/20221112-233420-ladsgroup.json
  • curprev 01:2101:21, 12 November 2022imported>Stashbot 848,236 bytes +62,212 marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2169:3317', diff saved to https://phabricator.wikimedia.org/P39331 and previous config saved to /var/cache/conftool/dbconfig/20221112-012122-marostegui.json

11 November 2022

  • curprev 01:2201:22, 11 November 2022imported>Stashbot 786,024 bytes +125,544 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2168:3317', diff saved to https://phabricator.wikimedia.org/P39163 and previous config saved to /var/cache/conftool/dbconfig/20221111-012250-ladsgroup.json

10 November 2022

8 November 2022

  • curprev 22:0022:00, 8 November 2022imported>Stashbot 604,525 bytes +104,241 mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mw-debug: apply
  • curprev 01:2501:25, 8 November 2022imported>Stashbot 500,284 bytes +154,243 cmjohnson@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)

6 November 2022

5 November 2022

  • curprev 12:5612:56, 5 November 2022imported>Stashbot 345,478 bytes +537 mfossati@deploy1002: Finished deploy [airflow-dags/platform_eng@c849762]: (no justification provided) (duration: 00m 49s)

4 November 2022

  • curprev 18:3118:31, 4 November 2022imported>Stashbot 344,941 bytes +23,994 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS buster

3 November 2022

  • curprev 22:4522:45, 3 November 2022imported>Stashbot 320,947 bytes +90,964 andrew@deploy1002: Finished deploy [horizon/deploy@9d02cd6]: (no justification provided) (duration: 01m 00s)

2 November 2022

  • curprev 23:2523:25, 2 November 2022imported>Stashbot 229,983 bytes +129,100 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2177 (T318605)', diff saved to https://phabricator.wikimedia.org/P37874 and previous config saved to /var/cache/conftool/dbconfig/20221102-232540-ladsgroup.json
  • curprev 01:1901:19, 2 November 2022imported>Stashbot 100,883 bytes −1,663,965 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P37541 and previous config saved to /var/cache/conftool/dbconfig/20221102-011937-ladsgroup.json

31 October 2022

  • curprev 22:2322:23, 31 October 2022imported>Stashbot 1,764,848 bytes +82,444 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance

29 October 2022

28 October 2022

  • curprev 20:4220:42, 28 October 2022imported>Stashbot 1,681,955 bytes +34,207 mutante: clouddumps* - deployed gerrit:848444 - as kind of expected it fails - most likely the project dirs are not automatically created before rsync runs the first time - T57503
  • curprev 01:1501:15, 28 October 2022imported>Stashbot 1,647,748 bytes +124,757 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1135 (T318950)', diff saved to https://phabricator.wikimedia.org/P36942 and previous config saved to /var/cache/conftool/dbconfig/20221028-011505-ladsgroup.json

27 October 2022

  • curprev 01:1601:16, 27 October 2022imported>Stashbot 1,522,991 bytes +90,843 sukhe@cumin2002: START - Cookbook sre.hosts.reimage for host cp4043.ulsfo.wmnet with OS buster

25 October 2022

  • curprev 22:3322:33, 25 October 2022imported>Stashbot 1,432,148 bytes +92,794 robh@cumin2002: START - Cookbook sre.hosts.provision for host cp4052.mgmt.ulsfo.wmnet with reboot policy FORCED
  • curprev 01:0901:09, 25 October 2022imported>Stashbot 1,339,354 bytes +82,471 ladsgroup@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db2169:3317 (T321312)', diff saved to https://phabricator.wikimedia.org/P36164 and previous config saved to /var/cache/conftool/dbconfig/20221025-010943-ladsgroup.json

22 October 2022

  • curprev 03:1603:16, 22 October 2022imported>Stashbot 1,256,883 bytes +83 bking@cumin2002: END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99)
  • curprev 00:0300:03, 22 October 2022imported>Stashbot 1,256,800 bytes +69,045 ladsgroup@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance

21 October 2022

  • curprev 01:1401:14, 21 October 2022imported>Stashbot 1,187,755 bytes +85,034 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db2138:3314 (T321312)', diff saved to https://phabricator.wikimedia.org/P35798 and previous config saved to /var/cache/conftool/dbconfig/20221021-011452-ladsgroup.json

19 October 2022

  • curprev 23:3323:33, 19 October 2022imported>Stashbot 1,102,721 bytes +35,546 wfan: civicrm upgraded from 477323fe to c96dd3ae
  • curprev 01:2001:20, 19 October 2022imported>Stashbot 1,067,175 bytes +11,210 cmjohnson@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1011.eqiad.wmnet with reason: host reimage

17 October 2022

  • curprev 23:1623:16, 17 October 2022imported>Stashbot 1,055,965 bytes +26,027 bblack@puppetmaster2001: conftool action : set/pooled=yes; selector: service=git-ssh

15 October 2022

  • curprev 23:2723:27, 15 October 2022imported>Stashbot 1,029,938 bytes +932 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depool db1131 T320879', diff saved to https://phabricator.wikimedia.org/P35497 and previous config saved to /var/cache/conftool/dbconfig/20221015-232716-ladsgroup.json

14 October 2022

  • curprev 22:5622:56, 14 October 2022imported>Stashbot 1,029,006 bytes +9,465 mutante: pcc-worker1003.puppet-diffs.eqiad1.wikimedia.cloud - out of disk space again - deleted 3.5GB job "1460" to unblock puppet compiling
(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)