You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Server Admin Log"

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View (newer 20 | ) (20 | 50 | 100 | 250 | 500)
  • curprev 00:51, 8 December 2021imported>Stashbot 461,391 bytes +29,464 ebernhardson@deploy1002: Synchronized php-1.38.0-wmf.12/extensions/GrowthExperiments/includes/NewcomerTasks/AddImage/AddImageSubmissionHandler.php: backport window for 744896 (duration: 01m 05s)
  • curprev 00:10, 7 December 2021imported>Stashbot 431,927 bytes +7,404 cwhite: end codfw opensearch upgrade T288621
  • curprev 01:14, 4 December 2021imported>Stashbot 424,523 bytes +12,137 mutante: mx2001 - did not come back from reboot, did not get IP on interface, could not start ferm, logged in via console with root password, in /etc/network/interfaces replaced all "ens5" with "ens13", rebooted again, selected previous kernel version
  • curprev 01:00, 3 December 2021imported>Stashbot 412,386 bytes +17,768 tgr@deploy1002: Synchronized php-1.38.0-wmf.9/extensions/GrowthExperiments: Backport: Add an image: Add test version of GEInfoboxTemplates (T291232) (duration: 00m 57s)
  • curprev 01:21, 2 December 2021imported>Stashbot 394,618 bytes +24,122 ryankemper: T280001 Rolling restart of low-traffic pybal hosts complete. All of `wcqs` is pooled and the pybal / ipvs related alerts have cleared
  • curprev 00:35, 1 December 2021imported>Stashbot 370,496 bytes +9,705 mwdebug-deploy@deploy1002: helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' .
  • curprev 00:22, 30 November 2021imported>Stashbot 360,791 bytes +17,004 mwdebug-deploy@deploy1002: helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' .
  • curprev 17:14, 28 November 2021imported>Stashbot 343,787 bytes +293 elukey@deploy1002: Finished deploy [ores/deploy@69ed061]: Canary upgrade of mwparserfromhell - T296563 (duration: 02m 11s)
  • curprev 19:55, 27 November 2021imported>Stashbot 343,494 bytes +1,043 andrew@deploy1002: Finished deploy [horizon/deploy@6115b3b]: network UI updates for T296548 (duration: 04m 14s)
  • curprev 16:11, 26 November 2021imported>Stashbot 342,451 bytes +4,657 arnoldokoth: drain kubestage1002 node in prep for decommissioning
  • curprev 20:44, 25 November 2021imported>Stashbot 337,794 bytes +25,467 ladsgroup@cumin1001: dbctl commit (dc=all): 'Depooling db1160 (T296143)', diff saved to https://phabricator.wikimedia.org/P17872 and previous config saved to /var/cache/conftool/dbconfig/20211125-204357-ladsgroup.json
  • curprev 00:37, 25 November 2021imported>Stashbot 312,327 bytes +56,025 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host elastic2066.codfw.wmnet with OS buster
  • curprev 00:25, 24 November 2021imported>Stashbot 256,302 bytes +24,082 pt1979@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs2012.codfw.wmnet with OS buster
  • curprev 00:51, 23 November 2021imported>Stashbot 232,220 bytes +27,289 urbanecm@deploy1002: Started scap: 69aa4a7: 7c0e074: Revert "Create redirect Special Pages for delete and protect action" (T295611; T296203; 4/4)
  • curprev 13:17, 21 November 2021imported>Stashbot 204,931 bytes +326 dcausse: restarting blazegraph on wdqs1007 (jvm stuck for 10h)
  • curprev 01:02, 20 November 2021imported>Stashbot 204,605 bytes +91 mutante: lists1001 - restarted apache, icinga alerts for the web UI, but recovered
  • curprev 00:27, 20 November 2021imported>Stashbot 204,514 bytes +9,398 cdanis@cumin1001: END (PASS) - Cookbook sre.network.cf (exit_code=0)
  • curprev 01:45, 19 November 2021imported>Stashbot 195,116 bytes +21,872 mutante: I think git-ssh6_22 is down (see alerts lvs2008/2009) due to the v6 issue from ongoing lvs maintenance. depooled in conftool
  • curprev 01:47, 18 November 2021imported>Stashbot 173,244 bytes +19,375 legoktm@cumin1001: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thumbor2005.codfw.wmnet
  • curprev 00:19, 17 November 2021imported>Stashbot 153,869 bytes +12,582 ryankemper: T276198 `ryankemper@cumin1001:~$ sudo cumin -b 3 '*elastic*' 'sudo run-puppet-agent --force'` Change looks good (no complaints from systemd), rolling out to rest of fleet / reenabling puppet
(newest | oldest) View (newer 20 | ) (20 | 50 | 100 | 250 | 500)