You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Server Admin Log"

Jump to navigation Jump to search

Diff selection: Mark the radio boxes of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)
  • curprev 21:49, 17 June 2021imported>Stashbot 462,242 bytes +15,302 legoktm: regenerating pipermail redirects to skip those with duplicate message-ids (T280731)
  • curprev 21:35, 16 June 2021imported>Stashbot 446,940 bytes +4,876 legoktm@deploy1002: helmfile [eqiad] Ran 'sync' command on namespace 'shellbox' for release 'main' .
  • curprev 17:54, 15 June 2021imported>Stashbot 442,064 bytes +4,022 dancy: testing upcoming Scap release on beta
  • curprev 00:37, 15 June 2021imported>Stashbot 438,042 bytes +26,468 eileen: civicrm revision changed from 31d07115a0 to 28ace1b86f, config revision is 2aed6ff89b
  • curprev 13:49, 12 June 2021imported>Stashbot 411,574 bytes +317 rzl@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 6 hosts with reason: alert noise, no impact, x2 is unused
  • curprev 23:37, 11 June 2021imported>Stashbot 411,257 bytes +6,591 mutante: removing firewall hole for mgmt networks to install* because it turned out it cant be used for firmware upgrades
  • curprev 01:10, 11 June 2021imported>Stashbot 404,666 bytes +17,032 eileen: process-control config revision is 2aed6ff89b
  • curprev 22:12, 9 June 2021imported>Stashbot 387,634 bytes +19,207 dzahn@cumin1001: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh1002.wikimedia.org
  • curprev 00:37, 9 June 2021imported>Stashbot 368,427 bytes +17,582 catrope@deploy1002: Synchronized wmf-config/InitialiseSettings.php: Config: Enable Wikisource OCR on select Wikisources (T283898) (duration: 01m 31s)
  • curprev 21:26, 7 June 2021imported>Stashbot 350,845 bytes +11,147 otto@cumin1001: END (PASS) - Cookbook sre.kafka.roll-restart-brokers (exit_code=0)
  • curprev 16:16, 5 June 2021imported>Stashbot 339,698 bytes +307 Amir1: deleting all private archives of mm2. All are inaccessible now (T282303)
  • curprev 00:21, 5 June 2021imported>Stashbot 339,391 bytes +11,476 mutante: backup1001 - systemctl baclua-dir works again (restoring backup for non-existing host)
  • curprev 00:12, 4 June 2021imported>Stashbot 327,915 bytes +24,727 ryankemper@cumin2002: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0)
  • curprev 00:40, 3 June 2021imported>Stashbot 303,188 bytes +15,354 ladsgroup@deploy1002: Synchronized php-1.37.0-wmf.7/extensions/Gadgets: Backport: Reduce message parse in GadgetHooks::getPreferences (second time) (T58633 T278650), Try II (duration: 00m 57s)
  • curprev 21:09, 1 June 2021imported>Stashbot 287,834 bytes +5,348 andrewbogott: dropping a bunch of tables from the labswiki db as per T284108
  • curprev 00:46, 1 June 2021imported>Stashbot 282,486 bytes +974 legoktm@deploy1002: Synchronized logos/config.yaml: Revert "Use eswiki 20th anniversary logos" (T280908) (duration: 01m 07s)
  • curprev 14:44, 29 May 2021imported>Stashbot 281,512 bytes +194 elukey: execute apt-get clean on an-airflow1001 to free space
  • curprev 08:06, 28 May 2021imported>Stashbot 281,318 bytes +368 oblivian@cumin1001: conftool action : set/pooled=inactive; selector: name=wdqs1003.eqiad.wmnet,dc=eqiad
  • curprev 23:56, 27 May 2021imported>Stashbot 280,950 bytes +10,488 robh@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab1004.eqiad.wmnet with reason: REIMAGE
  • curprev 23:07, 26 May 2021imported>Stashbot 270,462 bytes +14,277 ladsgroup@deploy1002: Synchronized php-1.37.0-wmf.7/includes/resourceloader/dependencystore/SqlModuleDependencyStore.php: Backport: resourceloader: Avoid primary connection in SqlModuleDependencyStore (2) (duration: 01m 06s)
  • curprev 00:47, 26 May 2021imported>Stashbot 256,185 bytes +10,892 eileen: civicrm revision changed from 584b96452a to eac772e9c9, config revision is 2ca92c3c3c
  • curprev 00:48, 25 May 2021imported>Stashbot 245,293 bytes +11,934 legoktm@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
  • curprev 14:25, 23 May 2021imported>Stashbot 233,359 bytes +262 urbanecm@deploy1002: Synchronized wmf-config/InitialiseSettings.php: EMERGENCY: f752f8b80c57a5b7e41b91873b3eac388535ac46: enwiktionary: Raise AF emergency disable treshold+count (T283460) (duration: 00m 57s)
  • curprev 22:13, 22 May 2021imported>Stashbot 233,097 bytes +265 legoktm: reset 2FA for User:Yuvipanda on wikitech
  • curprev 22:32, 21 May 2021imported>Stashbot 232,832 bytes +12,510 bstorm: upload nfsd-ldap: 1.2+deb10u1 to buster-wikimedia T283385
  • curprev 21:45, 20 May 2021imported>Stashbot 220,322 bytes +13,158 herron@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
  • curprev 01:01, 20 May 2021imported>Stashbot 207,164 bytes +12,243 mutante: signing puppet certs for doh2001 and doh2002.wikimedia.org (T283192)
  • curprev 18:40, 18 May 2021imported>Stashbot 194,921 bytes +14,463 razzi@deploy1002: Finished deploy [analytics/refinery@9392f1d] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@9392f1db6e66975304c8e9b2b7031acd3ed87fa7] (duration: 05m 16s)
  • curprev 00:56, 18 May 2021imported>Stashbot 180,458 bytes +19,706 eileen: civicrm revision changed from 38ac15233f to b3fb3c9cb0, config revision is 1f8d0a6bfa
  • curprev 00:13, 17 May 2021imported>Stashbot 160,752 bytes +408 legoktm: installing python3-dbg on lists1001
  • curprev 00:53, 16 May 2021imported>Stashbot 160,344 bytes +324 legoktm: restarted mailman3-web on lists1001, uwsgi looked like it got stuck, consuming all CPU/memory
  • curprev 20:42, 14 May 2021imported>Stashbot 160,020 bytes +3,428 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts people1002.eqiad.wmnet
  • curprev 00:39, 14 May 2021imported>Stashbot 156,592 bytes +8,035 ryankemper: T280382 `sudo -i wmf-auto-reimage-host -p T280382 --new wdqs2003.codfw.wmnet` on `ryankemper@cumin2001` tmux session `wdqs_reimage`
  • curprev 23:48, 12 May 2021imported>Stashbot 148,557 bytes +16,709 urbanecm@deploy1002: Synchronized php-1.37.0-wmf.4/extensions/WikiEditor/includes/WikiEditorHooks.php: 2f6af514c49d47bbec5ce51f9f7263015e039003? PHP VisualEditorFeatureUse logging: properly record session id (T281409) (duration: 01m 07s)
  • curprev 01:42, 12 May 2021imported>Stashbot 131,848 bytes +17,502 dzahn@cumin1001: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 1 day, 0:00:00 on people2002.codfw.wmnet with reason: new host
  • curprev 23:38, 10 May 2021imported>Stashbot 114,346 bytes +15,358 urbanecm@deploy1002: Synchronized wmf-config/CommonSettings.php: 779fb53bfd7a4d9b11f865df14f8a72adb97f33b: Update messages used for tech CoC (T280886) (duration: 00m 56s)
  • curprev 21:44, 9 May 2021imported>Stashbot 98,988 bytes −1,654,362 legoktm: restarted mailman3 again (T282348) pymysql.err.InternalError: (1205, 'Lock wait timeout exceeded; try restarting transaction')
  • curprev 17:18, 8 May 2021imported>Stashbot 1,753,350 bytes +105 Amir1: starting upgrade of batch G of mailing lists (T280322)
  • curprev 21:40, 7 May 2021imported>Stashbot 1,753,245 bytes +7,667 legoktm: deleted education@ from MM3, didn't import properly
  • curprev 23:50, 6 May 2021imported>Stashbot 1,745,578 bytes +25,003 brennen@deploy1002: rebuilt and synchronized wikiversions files: Rollback group1 and group2 to 1.37.0-wmf.3 (T282193)
  • curprev 00:35, 6 May 2021imported>Stashbot 1,720,575 bytes +28,723 Amir1: sudo service mailman3-web restart
  • curprev 01:45, 5 May 2021imported>Stashbot 1,691,852 bytes +24,478 ryankemper: T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs1006.eqiad.wmnet --dest wdqs1011.eqiad.wmnet --reason "transferring fresh wikidata journal following reimage" --blazegraph_instance blazegraph` on `ryankemper@cumin1001` tmux session `reimage`
  • curprev 23:18, 3 May 2021imported>Stashbot 1,667,374 bytes +11,155 urbanecm@deploy1002: Synchronized wmf-config/InitialiseSettings.php: 230ef5716b34ca83348667f289180313b76ce8a3: Prepare for new configuration option (T277951) (duration: 00m 57s)
  • curprev 13:40, 2 May 2021imported>Stashbot 1,656,219 bytes +311 dcaro@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on cloudmetrics1002.eqiad.wmnet with reason: Flaky host
  • curprev 19:12, 1 May 2021imported>Stashbot 1,655,908 bytes +557 Urbanecm: Invalidate password for MaraBot@SUL (T281586)
  • curprev 21:55, 30 April 2021imported>Stashbot 1,655,351 bytes +6,117 mutante: people1003 - rsycncing /home from peopel1002
  • curprev 01:08, 30 April 2021imported>Stashbot 1,649,234 bytes +23,564 ryankemper@cumin1001: END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) reboot without plugin upgrade (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw reboot - ryankemper@cumin1001 - T280563
  • curprev 00:41, 29 April 2021imported>Stashbot 1,625,670 bytes +24,040 tstarling@deploy1002: Synchronized php-1.37.0-wmf.3/includes/specials/pagers/ImageListPager.php: T281405 (duration: 01m 08s)
  • curprev 01:06, 28 April 2021imported>Stashbot 1,601,630 bytes +35,800 robh@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
  • curprev 23:28, 26 April 2021imported>Stashbot 1,565,830 bytes +16,596 mutante: renewing TLS cert for peopleweb.discovery.wmnet, adding *3 hosts
(newest | oldest) View (newer 50 | ) (20 | 50 | 100 | 250 | 500)