You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Server Admin Log"

Jump to navigation Jump to search

Diff selection: Mark the radio boxes of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)
  • curprev 01:12, 26 January 2020imported>Stashbot 141,710 bytes +333 cdanis: deployed
  • curprev 22:31, 24 January 2020imported>Stashbot 141,377 bytes +2,319 mutante: ganeti1003 - sudo gnt-instance remove etherpad1001.eqiad.wmnet (T224580)
  • curprev 00:46, 24 January 2020imported>Stashbot 139,058 bytes +9,927 mutante: cp4032 - starting varnishmtail.service
  • curprev 01:40, 23 January 2020imported>Stashbot 129,131 bytes +8,387 jforrester@deploy1001: Synchronized php-1.35.0-wmf.16/extensions/AbuseFilter/includes/AFComputedVariable.php: T243469 When no registration date is recorded, use 2008-01-15 (duration: 01m 08s)
  • curprev 01:13, 22 January 2020imported>Stashbot 120,744 bytes +8,885 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: resync, the last sync only took on half the appservers (duration: 01m 05s)
  • curprev 16:14, 20 January 2020imported>Stashbot 111,859 bytes +5,655 Urbanecm: Change email assigned to User:Sadsadas (T243222)
  • curprev 12:02, 19 January 2020imported>Stashbot 106,204 bytes +350 marostegui@cumin1001: dbctl commit (dc=all): 'Depool db2085:3311, db2085:3318 T243148', diff saved to https://phabricator.wikimedia.org/P10210 and previous config saved to /var/cache/conftool/dbconfig/20200119-120236-marostegui.json
  • curprev 00:46, 19 January 2020imported>Stashbot 105,854 bytes +429 cdanis: T238305 cp3053.mgmt /admin1-> racadm serveraction hardreset
  • curprev 21:56, 17 January 2020imported>Stashbot 105,425 bytes +5,582 urandom: bootstrapping restbase2023-c — T243000
  • curprev 00:45, 17 January 2020imported>Stashbot 99,843 bytes +13,373 mutante: urldownloaders - rm /etc/logrotate.d/squid3 ; systemctl start logrotate (this fixes failed logrotate because of squid3 vs squid file = duplicate entry, but puppet will recreate it)
  • curprev 00:40, 16 January 2020imported>Stashbot 86,470 bytes +11,747 bstorm_: set max_connections on db1133 (m5-master) back to 500 since the neutron connections seem fairly stable now T242817
  • curprev 00:22, 15 January 2020imported>Stashbot 74,723 bytes +6,560 bstorm_: restarted maintain-dbusers on labstore1004 after recovering the m5 DB's connection issue
  • curprev 01:03, 14 January 2020imported>Stashbot 68,163 bytes +7,915 catrope@deploy1001: Synchronized php-1.35.0-wmf.14/extensions/GrowthExperiments/: Various topic search-related cherry-picks (duration: 00m 57s)
  • curprev 00:58, 13 January 2020imported>Stashbot 60,248 bytes +1,047 jiji@cumin1001: conftool action : set/pooled=yes; selector: name=cp3061.esams.wmnet
  • curprev 05:34, 11 January 2020imported>Stashbot 59,201 bytes +242 volker-e@deploy1001: Finished deploy [design/style-guide@6a44c69]: Deploy design/style-guide: (duration: 00m 08s)
  • curprev 22:33, 10 January 2020imported>Stashbot 58,959 bytes +3,964 mutante: ms-be1026 sudo systemctl reset-failed (failed Session 372989 of user debmonitor)
  • curprev 00:45, 10 January 2020imported>Stashbot 54,995 bytes +8,040 catrope@deploy1001: Synchronized php-1.35.0-wmf.14/extensions/GrowthExperiments/: Expose tasktype/topic API parameter info (T240512) (duration: 01m 01s)
  • curprev 00:33, 9 January 2020imported>Stashbot 46,955 bytes +13,245 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: (no-op) set config page for newcomer tasks (T233465) (duration: 01m 05s)
  • curprev 00:38, 8 January 2020imported>Stashbot 33,710 bytes +15,858 ebernhardson@deploy1001: Finished deploy [wikimedia/discovery/analytics@024488f]: airflow: set mjolnir dag start date to today (20200108) (duration: 00m 42s)
  • curprev 00:59, 7 January 2020imported>Stashbot 17,852 bytes +9,482 mutante: ganeti - creating urldownloader1002.wikimedia.org in eqiad_C with 1 CPU, 1 GB RAM, 10 GB disk, public IP (T241979)
  • curprev 00:06, 6 January 2020imported>Stashbot 8,370 bytes +507 effie: pool cp3065 T238305
  • curprev 16:34, 4 January 2020imported>Stashbot 7,863 bytes +313 aborrero@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 22:14, 3 January 2020imported>Stashbot 7,550 bytes +3,165 volker-e@deploy1001: Finished deploy [design/style-guide@8054026]: Deploy design/style-guide: (duration: 00m 08s)
  • curprev 23:33, 2 January 2020imported>Stashbot 4,385 bytes +4,006 ejegg: updated Fundraising CiviCRM from d534f4e966 to 217a1f8c63
  • curprev 00:22, 2 January 2020imported>Stashbot 379 bytes +255 ejegg: re-enabled fundraising cron jobs
  • curprev 00:37, 1 January 2020imported>Nhatminh01 124 bytes −868,705 archive
  • curprev 16:00, 30 December 2019imported>Stashbot 868,829 bytes +235 ema: cp1083: restart ats-tls and varnish-fe after crashes - T241593
  • curprev 22:29, 29 December 2019imported>Stashbot 868,594 bytes +525 gehel: repooling wdqs1007
  • curprev 22:54, 28 December 2019imported>Stashbot 868,069 bytes +602 ebernhardson@deploy1001: Synchronized wmf-config/InitialiseSettings.php: cirrus: re-sync move more_like traffic to codfw (duration: 00m 54s)
  • curprev 19:51, 27 December 2019imported>Stashbot 867,467 bytes +696 gehel: restarted blazegraph on wdqs1007
  • curprev 10:12, 25 December 2019imported>Stashbot 866,771 bytes +312 aborrero@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 15:33, 24 December 2019imported>Stashbot 866,459 bytes +1,211 gehel@cumin1001: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0)
  • curprev 00:13, 24 December 2019imported>Stashbot 865,248 bytes +2,405 volker-e@deploy1001: Finished deploy [design/style-guide@efc240b]: Deploy design/style-guide: (duration: 00m 07s)
  • curprev 16:48, 22 December 2019imported>Stashbot 862,843 bytes +468 bd808: Mastodon feed moved to https://mastodon.social/@wikimedia_sal (T52109)
  • curprev 00:10, 22 December 2019imported>Stashbot 862,375 bytes +612 bd808: Wikimedia SAL messages now available on Mastodon! https://fosstodon.org/@wikimedia_sal
  • curprev 01:44, 21 December 2019imported>Stashbot 861,763 bytes +8,163 reedy@deploy1001: update-interwiki-cache aborted: Update interwiki cache (duration: 00m 07s)
  • curprev 00:52, 20 December 2019imported>Stashbot 853,600 bytes +18,576 mholloway-shell@deploy1001: Synchronized php-1.35.0-wmf.11/extensions/MachineVision: Fix: Allow a single period in $basePath in maintenance scripts (duration: 00m 54s)
  • curprev 01:44, 19 December 2019imported>Stashbot 835,024 bytes +20,843 twentyafterfour: deploying phabricator update (tagged release/2019-12-19/1)
  • curprev 00:22, 18 December 2019imported>Stashbot 814,181 bytes +17,525 niharika29@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Enable Glent M0 for dewiki, enwiki and frwiki - T237365 (duration: 01m 02s)
  • curprev 23:28, 16 December 2019imported>Stashbot 796,656 bytes +15,656 ejegg: updated fundraising civicrm from 505c653da1 to b2d0b5d66d
  • curprev 22:50, 14 December 2019imported>Stashbot 781,000 bytes +92 hashar: Restarted Gerrit on gerrit2001 # T240763
  • curprev 22:35, 13 December 2019imported>Stashbot 780,908 bytes +3,189 cdanis: T229686 ✔️ cdanis@mwdebug1001.eqiad.wmnet /srv/mediawiki 🕠🍺 scap pull
  • curprev 00:32, 13 December 2019imported>Stashbot 777,719 bytes +8,893 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: GrowthExperiments: Begin "initiation test" for suggested edits (T238888) (duration: 00m 55s)
  • curprev 01:42, 12 December 2019imported>Stashbot 768,826 bytes +11,852 volker-e@deploy1001: Finished deploy [design/style-guide@481eaf6]: Deploy design/style-guide: (duration: 00m 07s)
  • curprev 00:54, 11 December 2019imported>Stashbot 756,974 bytes +9,287 eileen: rocess-control config revision is 3f60e8fe9e
  • curprev 23:04, 9 December 2019imported>Stashbot 747,687 bytes +6,022 XenoRyet: reenabled Ingenico Connect recurring charge job
  • curprev 20:40, 8 December 2019imported>Stashbot 741,665 bytes +605 ejegg: disabled Ingenico Connect recurring charge job
  • curprev 01:47, 8 December 2019imported>Stashbot 741,060 bytes +2,785 andrew@deploy1001: Started deploy [horizon/deploy@accbbd1]: (no justification provided)
  • curprev 00:59, 7 December 2019imported>Stashbot 738,275 bytes +8,615 andrew@deploy1001: Started deploy [horizon/deploy@0f70602]: (no justification provided)
  • curprev 00:47, 6 December 2019imported>Stashbot 729,660 bytes +15,748 catrope@deploy1001: Synchronized wmf-config/CommonSettings.php: Use PHP serialization with HMAC for Kask session serialization (T222099) (duration: 01m 01s)
  • curprev 00:49, 5 December 2019imported>Stashbot 713,912 bytes +15,602 dzahn@cumin1001: conftool action : set/pooled=inactive; selector: name=phab1001-vcs.eqiad.wmnet
  • curprev 00:49, 4 December 2019imported>Stashbot 698,310 bytes +15,609 ejegg: changed donations queue consumer and thank you mailer to use 3 minute cycles
  • curprev 01:42, 3 December 2019imported>Stashbot 682,701 bytes +128,943 dzahn@cumin1001: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99)
  • curprev 01:00, 3 December 2019imported>Stashbot 681,404 bytes +127,646 James_F: mwscript sql.php --wiki=sewikimedia php-1.35.0-wmf.8/extensions/Translate/sql/translate_{…}.sql T239091
  • curprev 00:54, 19 November 2019imported>Stashbot 553,758 bytes +8,716 ejegg: updated civicrm from 1f454aa69a to 2802bdd649
  • curprev 00:44, 18 November 2019imported>Stashbot 545,042 bytes +199 tstarling@deploy1001: Synchronized php-1.35.0-wmf.5/includes/Rest/Handler/PageHistoryCountHandler.php: fix extremely slow query T238378 (duration: 00m 59s)
  • curprev 20:27, 16 November 2019imported>Stashbot 544,843 bytes +277 ariel@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 22:14, 15 November 2019imported>Stashbot 544,566 bytes +7,139 jeh@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 23:03, 14 November 2019imported>Stashbot 537,427 bytes +12,247 mutante: restarting gerrit to ncrease defaultThreadPoolSize to 2
  • curprev 00:41, 14 November 2019imported>Stashbot 525,180 bytes +9,904 ebernhardson: T237849 Start CirrusSearch forceSearchIndex.php commonswiki 2019-10-20T00:00:00 - 2019-11-14T01:00:00 pushing into jobqueue
  • curprev 23:57, 12 November 2019imported>Stashbot 515,276 bytes +11,275 mholloway-shell@deploy1001: Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Fix: Do not return after inserting a single suggestion (duration: 00m 52s)
  • curprev 22:51, 11 November 2019imported>Stashbot 504,001 bytes +3,806 ema@puppetmaster1001: conftool action : set/pooled=no; selector: name=cp3065.esams.wmnet
  • curprev 20:25, 9 November 2019imported>Stashbot 500,195 bytes +323 reedy@deploy1001: Synchronized langlist-labs: T237823 (duration: 00m 54s)
  • curprev 01:07, 9 November 2019imported>Stashbot 499,872 bytes +7,974 volker-e@deploy1001: Finished deploy [design/style-guide@ef82b69]: Deploy design/style-guide: (duration: 00m 07s)
  • curprev 01:23, 8 November 2019imported>Stashbot 491,898 bytes +10,234 reedy@deploy1001: Synchronized php-1.35.0-wmf.5/extensions/GlobalBlocking/: Prevent some extra db queries (duration: 00m 53s)
  • curprev 00:21, 7 November 2019imported>Stashbot 481,664 bytes +9,074 XioNoX: enable interface damping on primary eqsin-codfw link - T236878
  • curprev 00:07, 6 November 2019imported>Stashbot 472,590 bytes +5,234 mdholloway: created table wikimedia_editor_tasks_edit_streak on x1/wikishared (T234956)
  • curprev 00:15, 5 November 2019imported>Stashbot 467,356 bytes +6,242 mutante: gerrit - restarting service to re-enable jgit gc (T217497)
  • curprev 03:54, 3 November 2019imported>Stashbot 461,114 bytes +719 andrew@deploy1001: Finished deploy [horizon/deploy@0c024d4]: one more prefix fix (duration: 03m 35s)
  • curprev 00:58, 2 November 2019imported>Stashbot 460,395 bytes +3,957 mutante: gerrit-replica - created missing /var/lib/gerrit2/review_site/tmp and restarted service - service back up on buster (T176774)
  • curprev 01:21, 1 November 2019imported>Stashbot 456,438 bytes +10,859 jforrester@deploy1001: Synchronized php-1.35.0-wmf.4/resources/src/mediawiki.widgets/mw.widgets.UsersMultiselectWidget.js: T236460 mw.widgets.UsersMultiselectWidget: Fix property name (duration: 00m 54s)
  • curprev 00:41, 31 October 2019imported>Stashbot 445,579 bytes +20,560 jforrester@deploy1001: Synchronized php-1.35.0-wmf.4/extensions/WikiLove/resources/ext.wikiLove.icon.vector.css: T236958 Fix Vector icon after upstream change (duration: 01m 02s)
  • curprev 23:23, 28 October 2019imported>Stashbot 425,019 bytes +8,544 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Deploy Echo kask migration to officewiki for testing, part 3 (T222851) (duration: 00m 52s)
  • curprev 11:31, 26 October 2019imported>Stashbot 416,475 bytes +128 XioNoX: restart cr3-esams
  • curprev 22:55, 25 October 2019imported>Stashbot 416,347 bytes +9,927 mutante: moscovium rm /dev/shm/envoy_shared_memory_0 to revive envoy which failed to run after changing ports and reinstalling it (T180641)
  • curprev 00:52, 25 October 2019imported>Stashbot 406,420 bytes +18,865 krinkle@deploy1001: Synchronized php-1.35.0-wmf.3/extensions/LiquidThreads/classes/View.php: (no justification provided) (duration: 00m 54s)
  • curprev 00:03, 24 October 2019imported>Stashbot 387,555 bytes +11,289 mutante: restarting gerrit to increase heap_size from 20G to 32G (T225166 T222391)
  • curprev 23:57, 22 October 2019imported>Stashbot 376,266 bytes +12,691 maxsem@deploy1001: Synchronized php-1.35.0-wmf.3/includes/block/DatabaseBlock.php: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/545373/ (duration: 00m 59s)
  • curprev 23:15, 21 October 2019imported>Stashbot 363,575 bytes +15,479 thcipriani: ops/puppet:sudo -u gerrit2 git update-ref refs/changes/66/535966/meta d6909e0537b7fe7ca6fb8989e22ed5e1c1e46e89 && sudo -u gerrit2 git update-ref refs/changes/66/535966/meta 8494c28eee163aaa48f6274e899e0e79f3a757f3 on gerrit1001
  • curprev 08:41, 19 October 2019imported>Stashbot 348,096 bytes +48 XioNoX: add user papaul to fasw-c-eqiad
  • curprev 00:05, 19 October 2019imported>Stashbot 348,048 bytes +12,443 mutante: LDAP - adding verenali to wmde and nda groups, to match raja_wmde (T233807, T231677)
  • curprev 21:42, 17 October 2019imported>Stashbot 335,605 bytes +9,045 mholloway-shell@deploy1001: Finished deploy [mobileapps/deploy@d663006]: Update mobileapps to f345673 (duration: 05m 38s)
  • curprev 00:55, 17 October 2019imported>Stashbot 326,560 bytes +13,865 dzahn@cumin1001: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0)
  • curprev 00:09, 16 October 2019imported>Stashbot 312,695 bytes +11,193 mutante: wikitech - make JBond a "content administrator" to give the ability to create server fingerprint pages
  • curprev 23:27, 14 October 2019imported>Stashbot 301,502 bytes +9,644 Krinkle: Delete 2019-09-01––2019-09-10 arclamp trace logs from webperf1002, and decompress the rest of 2019-09 (this will trigger svg re-generation), T235425
  • curprev 00:52, 13 October 2019imported>Stashbot 291,858 bytes +984 krinkle@deploy1001: Synchronized wmf-config/CommonSettings.php: ec77b1b515940c73 (duration: 00m 55s)
  • curprev 15:39, 11 October 2019imported>Stashbot 290,874 bytes +3,232 AndyRussG: updated fruec from 18d89675d0 to 1e6a6ee2de
  • curprev 22:27, 10 October 2019imported>Stashbot 287,642 bytes +12,182 jforrester@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Stop setting wgMFMobileFormatterHeadings, unread T232690 (duration: 00m 51s)
  • curprev 00:39, 10 October 2019imported>Stashbot 275,460 bytes +16,804 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.ipmi-password-reset (exit_code=0)
  • curprev 23:28, 8 October 2019imported>Stashbot 258,656 bytes +11,791 mutante: phab1001 - replacing tin.eqiad.wmnet with deploy1001.eqiad.wmnet in phabricator/deployment-cache/.config:git_server - wondering if we can ever get rid of tin (T190568)
  • curprev 00:33, 8 October 2019imported>Stashbot 246,865 bytes +11,958 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.ipmi-password-reset (exit_code=0)
  • curprev 20:11, 6 October 2019imported>Stashbot 234,907 bytes +365 Urbanecm: mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Racconish /home/urbanecm/T234741 (T234741)
  • curprev 06:48, 5 October 2019imported>Stashbot 234,542 bytes +151 elukey: force umount/remount of /mnt/hdfs on an-coord1001 - processes stuck in D state, fuser proc consuming a ton of memory
  • curprev 22:06, 4 October 2019imported>Stashbot 234,391 bytes +3,026 mutante: ms-be1020 - power cycle via mgmt - host down
  • curprev 23:50, 3 October 2019imported>Stashbot 231,365 bytes +5,107 mutante: gerrit - restarting for replication config tweaks
  • curprev 00:51, 3 October 2019imported>Stashbot 226,258 bytes +6,323 mutante: gerrit1001 - removing wrong IPv6 address from interface, running puppet
  • curprev 00:54, 2 October 2019imported>Stashbot 219,935 bytes +8,918 ejegg: updated fundraising CiviCRM from 6d90d0cf06 to 12c5727a23
  • curprev 00:12, 1 October 2019imported>Stashbot 211,017 bytes +7,795 mutante: phabricator - upgrading PHP version to 7.2.22 - T230024
  • curprev 16:28, 28 September 2019imported>Stashbot 203,222 bytes +77 vgutierrez: restarting acme-chief on acmechief1001
  • curprev 22:44, 27 September 2019imported>Stashbot 203,145 bytes +3,427 mutante: phab2001 - apt-get autoremove - remove unused python and ruby packages
  • curprev 00:53, 27 September 2019imported>Stashbot 199,718 bytes +16,143 twentyafterfour: hotfixing phabricator fatal exception refs T233998
  • curprev 21:59, 25 September 2019imported>Stashbot 183,575 bytes +17,154 bblack: remove GRE MTU hacks on archiva1001 gerrit2001 cobalt install1002 - T232602
  • curprev 01:00, 25 September 2019imported>Stashbot 166,421 bytes +13,033 krinkle@deploy1001: Synchronized wmf-config/InitialiseSettings.php: 3373247e123b53 - create new file (duration: 01m 05s)
  • curprev 00:43, 24 September 2019imported>Stashbot 153,388 bytes +8,330 mutante: db2060 - remove PXE flag boot override - set Boot Device to none
  • curprev 19:07, 22 September 2019imported>Stashbot 145,058 bytes +53 volans|off: marostegui set s3 master RW
  • curprev 05:42, 21 September 2019imported>Stashbot 145,005 bytes +493 shdubsh: re-enable input-kafka-rsyslog-shipper in codfw
  • curprev 01:34, 21 September 2019imported>Stashbot 144,512 bytes +5,777 mutante: restarting mobileapps service on scb*
  • curprev 23:23, 19 September 2019imported>Stashbot 138,735 bytes +9,963 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 23:18, 18 September 2019imported>Stashbot 128,772 bytes +12,473 krinkle@deploy1001: Synchronized php-1.34.0-wmf.23/extensions/MobileFrontend/resources/dist/: T233260, 1667ed957a19067 (duration: 01m 04s)
  • curprev 23:26, 17 September 2019imported>Stashbot 116,299 bytes +14,935 twentyafterfour@deploy1001: rebuilt and synchronized wikiversions files: group0 wikis to 1.34.0-wmf.23 refs T220748
  • curprev 00:34, 17 September 2019imported>Stashbot 101,364 bytes +15,480 AndyRussG: updated fruec from fb29cb7407 to 97128874bf
  • curprev 16:51, 15 September 2019imported>Krinkle 85,884 bytes +360
  • curprev 23:42, 14 September 2019imported>Stashbot 85,524 bytes +133 onimisionipe: force shard allocation (dewiki_content_1566659363[4]) on eqiad cluster
  • curprev 01:14, 14 September 2019imported>Stashbot 85,391 bytes +7,579 ejegg: updated fundraising python tools from 1e405864d7 to e1b81688c6
  • curprev 23:35, 12 September 2019imported>Stashbot 77,812 bytes +9,962 XioNoX: enable netflow sampling on cr1-codfw
  • curprev 23:50, 11 September 2019imported>Stashbot 67,850 bytes +13,999 ejegg: updated payments-wiki from 5432f9c3a4 to 4ebbdb247d
  • curprev 20:46, 10 September 2019imported>Stashbot 53,851 bytes +9,981 ejegg: updated payments-wiki from 15baf7f58b to 5432f9c3a4
  • curprev 00:59, 10 September 2019imported>Stashbot 43,870 bytes +5,338 krinkle@deploy1001: Finished deploy [performance/navtiming@f2a0863]: (no justification provided) (duration: 00m 05s)
  • curprev 22:04, 8 September 2019imported>Stashbot 38,532 bytes +153 eileen: civicrm revision changed from c3172daf95 to d1d65f37ea, config revision is 5a6a9c6c03
  • curprev 21:33, 6 September 2019imported>Stashbot 38,379 bytes +4,389 cdanis: cdanis@mw1317.eqiad.wmnet ~ 🕠🍺 sudo -i depool
  • curprev 23:13, 5 September 2019imported>Stashbot 33,990 bytes +8,363 ayounsi@deploy1001: Finished deploy [netbox/deploy@367ca84]: test (duration: 00m 42s)
  • curprev 00:55, 5 September 2019imported>Stashbot 25,627 bytes +10,215 jforrester@deploy1001: Synchronized wmf-config/CommonSettings.php: CommonSettings: Factor out write of variant config into MWConfigCacheGenerator, part 2 (duration: 00m 53s)
  • curprev 01:02, 4 September 2019imported>Stashbot 15,412 bytes +11,302 krinkle@deploy1001: Synchronized wmf-config/CommonSettings.php: ed5297c10 / T217830 (duration: 00m 59s)
  • curprev 22:08, 2 September 2019imported>Stashbot 4,110 bytes +3,236 ebernhardson: ban elastic1027 from production-search-chi
  • curprev 23:23, 1 September 2019imported>Nhatminh01 874 bytes −746,338 fix something
  • curprev 13:33, 31 August 2019imported>Stashbot 747,212 bytes +134 krinkle@deploy1001: Synchronized wmf-config/CommonSettings.php: 88ba4f8f4d49 (duration: 00m 55s)
  • curprev 22:30, 30 August 2019imported>Stashbot 747,078 bytes +4,405 ejegg: disabled fundraising targetsmart import jobs
  • curprev 23:23, 29 August 2019imported>Stashbot 742,673 bytes +4,768 ejegg: updated payments-wiki from 1d5d7503b0 to 51d9ed79b6
  • curprev 23:17, 28 August 2019imported>Stashbot 737,905 bytes +7,538 urbanecm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: 8bfe43a: Add scielo.br to wgCopyUploadsDomains for commonswiki (T231402) (duration: 00m 55s)
  • curprev 23:45, 27 August 2019imported>Stashbot 730,367 bytes +9,995 Urbanecm: Evening SWAT done
  • curprev 01:47, 27 August 2019imported>Stashbot 720,372 bytes +5,698 eileen: process-control config revision is 24aed9745e
  • curprev 13:46, 25 August 2019imported>Stashbot 714,674 bytes +176 volans: uploaded spicerack_0.0.27-1_amd64.deb to apt.wikimedia.org stretch-wikimedia
  • curprev 00:35, 25 August 2019imported>Stashbot 714,498 bytes +283 herron: set icinga downtimes on flapping cr2-eqiad and cr2-codfw alerts until monday
  • curprev 23:34, 23 August 2019imported>Stashbot 714,215 bytes +1,363 robh@cumin1001: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0)
  • curprev 23:41, 22 August 2019imported>Stashbot 712,852 bytes +5,069 urbanecm@deploy1001: Synchronized php-1.34.0-wmf.19/extensions/WikibaseLexeme: SWAT: e4a5457: Fix Lexemes RDF generation (T230974) (duration: 00m 49s)
  • curprev 00:37, 22 August 2019imported>Stashbot 707,783 bytes −393,674 XioNoX: run /usr/local/sbin/restart-php7.2-fpm on mwdebug1001/2
  • curprev 23:53, 20 August 2019imported>Stashbot 1,101,457 bytes +7,677 ebernhardson@deploy1001: Finished deploy [search/mjolnir/deploy@a7bf6cf]: bulk_daemon: Increase bulk request_timeout (duration: 03m 40s)
  • curprev 21:21, 19 August 2019imported>Stashbot 1,093,780 bytes +5,899 ejegg: updated payments-wiki from 7b8091ba87 to 85dce8f79f
  • curprev 08:28, 18 August 2019imported>Stashbot 1,087,881 bytes +165 onimisionipe: running `_cluster/reroute?pretty&explain=true&retry_failed` on eqiad production-search cluster to force allocation of shards
  • curprev 19:48, 16 August 2019imported>Stashbot 1,087,716 bytes +1,009 sbassett: Deployed security patch for T230576 (ex:MobileFrontend)
  • curprev 23:35, 15 August 2019imported>Stashbot 1,086,707 bytes +7,511 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@b4da6e4]: Rollback blazegraph due to T230588 (duration: 09m 48s)
  • curprev 00:15, 15 August 2019imported>Stashbot 1,079,196 bytes +5,064 robh: scs-ulsfo offline due to networking issues, rob returning tomorrow with fix T230077
  • curprev 20:43, 13 August 2019imported>Stashbot 1,074,132 bytes +11,563 ejegg: rolled back payments-wiki from 9ed8be0532 to 9533f70fab
  • curprev 23:24, 12 August 2019imported>Stashbot 1,062,569 bytes +5,587 XioNoX: add samplicator to buster-wikimedia repo
  • curprev 22:38, 11 August 2019imported>Stashbot 1,056,982 bytes +198 urbanecm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Temporary make account creation limits more restrictive (T230304) (duration: 00m 50s)
  • curprev 01:49, 10 August 2019imported>Stashbot 1,056,784 bytes +187 mutante: mwmaint - running (1 of 8, the one for en) refreshLinks maintenance cron manually to verify it works after switching mwscriptwikiset to PHP7.2 (T195392)
  • curprev 00:52, 10 August 2019imported>Stashbot 1,056,597 bytes +3,069 mutante: mwmaint - running update_flaggedrevs_stats - updates the flagged revs statistics table on each wiki
  • curprev 23:50, 8 August 2019imported>Stashbot 1,053,528 bytes +6,524 Urbanecm: Evening SWAT done
  • curprev 00:48, 8 August 2019imported>Stashbot 1,047,004 bytes +7,522 mutante: mwdebug2002 - sudo -i restart-php7.2-fpm
  • curprev 00:48, 7 August 2019imported>Stashbot 1,039,482 bytes +10,857 mutante: restarting gerrit to apply config change 528276 to exclude some projects from github replication
  • curprev 23:34, 5 August 2019imported>Stashbot 1,028,625 bytes +13,466 mutante: mwmaint1002 - remove getJobQueueLengths.php from www-data's crontab (T195392)
  • curprev 18:45, 4 August 2019imported>Stashbot 1,015,159 bytes +150 krinkle@deploy1001: Synchronized wmf-config/abusefilter.php: labs-only noop - f740f89c594979 (duration: 00m 50s)
  • curprev 12:02, 3 August 2019imported>Stashbot 1,015,009 bytes +321 gilles: purging ruwiki articles on mwmaint1002
  • curprev 00:40, 3 August 2019imported>Stashbot 1,014,688 bytes +5,626 ejegg: rolled back fundraising python tools from 493a38f9e0 to 2a56e5e283
  • curprev 23:32, 1 August 2019imported>Stashbot 1,009,062 bytes +8,682 Urbanecm: Evening SWAT done
  • curprev 00:33, 1 August 2019imported>Stashbot 1,000,380 bytes +12,622 krinkle@deploy1001: Synchronized php-1.34.0-wmf.16/resources/Resources.php: acfff6751f3b8f7650 (duration: 00m 54s)
  • curprev 00:24, 31 July 2019imported>Stashbot 987,758 bytes +10,243 eileen: tools revision changed from 4910f1507c to 2a56e5e283
  • curprev 23:37, 29 July 2019imported>Stashbot 977,515 bytes +9,325 XioNoX: replace export policy BGP_Wikimedia_own_space with BGP_Wikimedia_no_dfz in ams
  • curprev 15:13, 28 July 2019imported>Stashbot 968,190 bytes +99 arturo: disable 1m load average check in icinga for labstore1007 for 24h
  • curprev 17:39, 27 July 2019imported>Stashbot 968,091 bytes +223 bd808: Updated profile & images for @wikimediatech twitter account
  • curprev 00:35, 27 July 2019imported>Stashbot 967,868 bytes +4,085 mutante: restbase-dev1006 - starting nagios-nrpe-server
  • curprev 00:53, 26 July 2019imported>Stashbot 963,783 bytes +13,853 ejegg: re-enabled dedupe_civicrm_contacts and major_gifts_addresses fundraising jobs
  • curprev 23:46, 24 July 2019imported>Stashbot 949,930 bytes +10,601 nuria@deploy1001: Finished deploy [analytics/refinery@7d93398]: deploying refinery 0.0.96 (skipping 0.0.95 due to some jenkins/archiva issues). Try 2 (duration: 13m 34s)
  • curprev 23:44, 23 July 2019imported>Stashbot 939,329 bytes +9,022 eileen: civicrm revision changed from 88e9f24893 to 875ab97742, config revision is 4006d3bdc5
  • curprev 00:06, 23 July 2019imported>Stashbot 930,307 bytes +11,599 Urbanecm: slwiki updateCollection.php completed (T208984)
  • curprev 01:06, 21 July 2019imported>Stashbot 918,708 bytes +80 Urbanecm: Deployed patch for T228574
  • curprev 22:36, 19 July 2019imported>Stashbot 918,628 bytes +5,701 mutante: phab2001 - switching apache to php-fpm and worker instead of mpm-prefork (to match phab1001) (T190568 T137928 T190572)
  • curprev 00:52, 19 July 2019imported>Stashbot 912,927 bytes +9,770 sbassett@deploy1001: Synchronized private/PrivateSettings.php: Add even more severe rate limits for eswikiquote and some other, smaller wikis (T227416) (duration: 00m 58s)
  • curprev 23:51, 17 July 2019imported>Stashbot 903,157 bytes +11,329 catrope@deploy1001: Synchronized wmf-config/CommonSettings.php: Add wmgUseTheWikipediaLibrary (false everywhere, no-op) (duration: 00m 54s)
  • curprev 00:50, 17 July 2019imported>Stashbot 891,828 bytes +10,253 mutante: wikitech-static commented out cert renewal cron job out of caution - still needs fixing but continue tomorrow
  • curprev 00:24, 16 July 2019imported>Stashbot 881,575 bytes +12,943 krinkle@deploy1001: Synchronized php-1.34.0-wmf.13/includes/cache/LinkCache.php: 4a5f4ca2fd788 (duration: 00m 51s)
  • curprev 13:18, 14 July 2019imported>Stashbot 868,632 bytes +257 godog: silence mr1-eqsin.oob IPv6 until tomorrow 8 UTC - T227967
  • curprev 01:51, 13 July 2019imported>Stashbot 868,375 bytes +68 MaxSem: DIsabled 2FA for my staff account
  • curprev 23:35, 12 July 2019imported>Stashbot 868,307 bytes +3,936 mutante: netmon1003 - shutdown -h now after it's gone from Icinga now
  • curprev 01:01, 12 July 2019imported>Stashbot 864,371 bytes +11,144 Krinkle: mw1342 generated some ~ 11,500 additional PHP errors over a 4 hour period (18:00-22:30 UTC), ref T224491
  • curprev 23:16, 10 July 2019imported>Stashbot 853,227 bytes +7,815 jforrester@deploy1001: Synchronized php-1.34.0-wmf.13/extensions/CirrusSearch/includes: T227691 RedirectsAndIncomingLinks: succeede or fail, but not both (duration: 01m 02s)
  • curprev 23:07, 9 July 2019imported>Stashbot 845,412 bytes +8,973 jhuneidi@deploy1001: rebuilt and synchronized wikiversions files: group0 wikis to 1.34.0-wmf.13 refs T220738
  • curprev 01:47, 9 July 2019imported>Stashbot 836,439 bytes +12,820 XioNoX: restart PHP FPM on mwdebug2001
  • curprev 20:13, 7 July 2019imported>Stashbot 823,619 bytes +210 urandom: decommissioning restbase1017-b -- T222960
  • curprev 07:56, 6 July 2019imported>Stashbot 823,409 bytes +74 thcipriani: restarting gerrit out of heap space
  • curprev 17:18, 5 July 2019imported>Stashbot 823,335 bytes +5,803 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@4b7cdf5]: Deploy new versions preparing for reload (duration: 00m 39s)
  • curprev 21:50, 4 July 2019imported>Stashbot 817,532 bytes +6,053 volans@deploy1001: Finished deploy [debmonitor/deploy@0ee26a3]: Deploy Debmonitor v0.1.10 (duration: 00m 48s)
  • curprev 00:41, 4 July 2019imported>Stashbot 811,479 bytes +18,775 twentyafterfour: phabricator upgrade complete
  • curprev 00:14, 3 July 2019imported>Stashbot 792,704 bytes +11,829 eileen: process-control config revision is 8e215d07f2 (renable jobs)
  • curprev 20:33, 1 July 2019imported>Stashbot 780,875 bytes +7,789 milimetric@deploy1001: Finished deploy [analytics/refinery@4e9894c]: minor, just removing hiwikisource from sqoop list (duration: 01m 33s)
  • curprev 23:27, 30 June 2019imported>Stashbot 773,086 bytes +199 urbanecm@deploy1001: Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 45s)
  • curprev 21:19, 28 June 2019imported>Stashbot 772,887 bytes +6,285 otto@deploy1001: Finished deploy [eventstreams/deploy@2af2719]: Manually blacklisting IP - T226808 (duration: 03m 07s)
  • curprev 01:22, 28 June 2019imported>Stashbot 766,602 bytes +7,465 Krinkle: Killing arclamp-log on webperf1002, no flame graphs for three days, presumably mwlog/redis connection dropped again. T215740
  • curprev 01:12, 27 June 2019imported>Stashbot 759,137 bytes +11,139 bblack: depool eqiad front edge
  • curprev 22:50, 25 June 2019imported>Stashbot 747,998 bytes +8,261 jforrester@deploy1001: Synchronized php-1.34.0-wmf.11/extensions/Echo/modules/nojs/: T226503 Fix badge icons in Monobook (duration: 00m 56s)
  • curprev 00:10, 25 June 2019imported>Stashbot 739,737 bytes +11,072 reedy@deploy1001: Synchronized wmf-config/flaggedrevs.php: poke at autopromote config (duration: 00m 54s)
  • curprev 14:54, 21 June 2019imported>Stashbot 728,665 bytes +5,320 jmm@cumin2001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)
  • curprev 00:45, 21 June 2019imported>Stashbot 723,345 bytes +15,806 RoanKattouw: Running FlowReserializeRevisionContent.php on testwiki
  • curprev 18:10, 19 June 2019imported>Stashbot 707,539 bytes +18,219 legoktm: added MatmaRex to extension-VisualEditor-staff Gerrit group
  • curprev 22:33, 18 June 2019imported>Stashbot 689,320 bytes +17,179 jijiki: pool thumbor1001
  • curprev 00:22, 18 June 2019imported>Stashbot 672,141 bytes +12,375 RoanKattouw: Running populateRevisionSha1.php on dewikivoyage for T219816
  • curprev 14:20, 16 June 2019imported>Stashbot 659,766 bytes +253 Urbanecm: running mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user='AKA MBG' /home/urbanecm/T225886
  • curprev 20:38, 15 June 2019imported>Stashbot 659,513 bytes +618 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@55174a4]: deploy new pattern for bots (duration: 21m 42s)
  • curprev 23:23, 14 June 2019imported>Stashbot 658,895 bytes +1,595 ejegg: updated payments-wiki from 75abd71cc1 to 79d1822644
  • curprev 00:11, 14 June 2019imported>Stashbot 657,300 bytes +1,216 SMalyshev: depooled wdqs1005 - let it catch up
  • curprev 00:45, 13 June 2019imported>Stashbot 656,084 bytes +2,359 paravoid: setting the CPU governor to performance for ms-be1036 (a while ago)
  • curprev 19:24, 11 June 2019imported>Stashbot 653,725 bytes +610 tzatziki: Removing four (4) files for legal compliance
  • curprev 19:39, 10 June 2019imported>Stashbot 653,115 bytes +2,118 thcipriani: restarting jenkins
  • curprev 08:30, 9 June 2019imported>Stashbot 650,997 bytes +79 vgutierrez: rebooting lvs4007 after NIC driver crash
  • curprev 11:58, 8 June 2019imported>Stashbot 650,918 bytes +203 godog: stop swift processes on ms-be1033 - T223518
  • curprev 18:56, 7 June 2019imported>Stashbot 650,715 bytes +6,617 herron: performing rolling reboots of logstash codfw frontends for security updates
  • curprev 00:00, 7 June 2019imported>Stashbot 644,098 bytes +674 jforrester@deploy1001: Synchronized wmf-config/CommonSettings.php: Remove unused preference T47877-buster (duration: 00m 47s)
  • curprev 21:01, 6 June 2019imported>Stashbot 643,424 bytes +8,375 bstorm_: T224850 depooled labsdb1011
  • curprev 22:15, 5 June 2019imported>Stashbot 635,049 bytes +20,643 chaomodus: restarting gerrit on cobalt due to it being down (seems like Java out of heap space)
  • curprev 00:09, 4 June 2019imported>Stashbot 614,406 bytes +7,967 bstorm_: T223406 repooled labsdb1011 after completing view updates
  • curprev 20:28, 2 June 2019imported>Stashbot 606,439 bytes +347 onimisionipe: pooled wdqs1007. It caught up on lag
  • curprev 22:49, 1 June 2019imported>Stashbot 606,092 bytes +604,365 krinkle@deploy1001: Synchronized php-1.34.0-wmf.7/extensions/3D/modules/mmv.3d.js: T224812 / bd4fbfddbe1a0 (duration: 01m 07s)
  • curprev 20:25, 3 February 2019imported>Stashbot 1,727 bytes +610 elukey: powercycle mw1272 - no ssh, no tty available via com2 - DIMM correctable errors + OEM errors registered in getsel
  • curprev 01:10, 3 February 2019imported>Stashbot 1,117 bytes −57,244 elukey: powercycle mw1299 - can't ssh nor get a tty via console - racadm getsel shows "An OEM diagnostic event occurred."
  • curprev 16:33, 13 January 2019imported>Stashbot 58,361 bytes +103 hoo: Updated operations/dumps/dcat (559dee37452..a86285f4e7) on snapshot1008
  • curprev 21:46, 12 January 2019imported>Stashbot 58,258 bytes +291 akosiaris: restart all zotero pods in eqiad
  • curprev 21:56, 11 January 2019imported>Stashbot 57,967 bytes +2,518 jforrester@deploy1001: Finished scap: Full scap sync to update wmf.12 i18n for the weekend Idf2a67860f (duration: 19m 12s)
  • curprev 00:50, 11 January 2019imported>Stashbot 55,449 bytes +11,158 XioNoX: bump prefix limit for AS6939 in eqsin
  • curprev 01:41, 10 January 2019imported>Stashbot 44,291 bytes +8,082 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Make GrowthExperiments config wmf.12-proof (duration: 00m 52s)
  • curprev 01:15, 9 January 2019imported>Stashbot 36,209 bytes +13,531 jforrester@deploy1001: Synchronized php-1.33.0-wmf.12/extensions/Wikibase/repo/RepoHooks.php: T213227 Don't have onApiCheckCanExecute die for inactive entity types (duration: 00m 53s)
  • curprev 00:22, 8 January 2019imported>Stashbot 22,678 bytes +7,597 gehel: restarting tilerator on all maps servers
  • curprev 14:50, 6 January 2019imported>Stashbot 15,081 bytes +251 ariel@deploy1001: Finished deploy [dumps/dumps@cb30b6c]: check xml files for closing mediawiki tag (duration: 00m 06s)
  • curprev 20:24, 5 January 2019imported>Elukey 14,830 bytes +143 →‎2019-01-05
  • curprev 23:07, 4 January 2019imported>Stashbot 14,687 bytes +1,714 mutante: scandium apt-get remove nodejs nodes-legacy ; puppet agent -tv - after merging gerrit:482150 this fixed "you have held broken packages" issue, now we are at a puppet dependecy cycle with apt::pin T201366
  • curprev 00:07, 4 January 2019imported>Stashbot 12,973 bytes +6,055 mutante: an-coord1001 - apt-get clean to free disk space, reacting to Icinga alert for running out of disk
  • curprev 00:21, 3 January 2019imported>Stashbot 6,918 bytes +6,798 mutante: notebook1004 - started nagios-nrpe-server one more time
  • curprev 00:56, 1 January 2019imported>Nhatminh01 120 bytes −133,599 archive December
  • curprev 07:17, 30 December 2018imported>Stashbot 133,719 bytes +223 elukey: restart pdfrender on scb1002 (alarms flapping)
  • curprev 14:53, 29 December 2018imported>Stashbot 133,496 bytes +935 bawolff@deploy1001: Synchronized wmf-config/InitialiseSettings.php: T212667 218371fd35 - Adjust mw.org abusefilter emergency shutoff threshold down to 0.3 (duration: 00m 46s)
  • curprev 18:14, 28 December 2018imported>Stashbot 132,561 bytes +537 bawolff@deploy1001: Synchronized wmf-config/InitialiseSettings.php: 97446843a27 T212667 - Temp increase abusefilter emergency cutoff on mw.org to deal with spam attack (duration: 00m 46s)
  • curprev 19:31, 27 December 2018imported>Stashbot 132,024 bytes +1,926 mobrovac@deploy1001: Finished deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays), take #3 (duration: 25m 49s)
  • curprev 16:37, 26 December 2018imported>Stashbot 130,098 bytes +468 godog: clear logstash persistent queue /var/lib/logstash on logstash100[789]
  • curprev 05:38, 24 December 2018imported>Stashbot 129,630 bytes +54 _joe_: powercycling lvs3001
  • curprev 11:53, 23 December 2018imported>Stashbot 129,576 bytes +330 marostegui: Fix replication on db1124:3318
  • curprev 18:45, 22 December 2018imported>Stashbot 129,246 bytes +294 elukey: manually clean up of old log files on an-coord1001 (disk space issues)
  • curprev 01:00, 22 December 2018imported>Stashbot 128,952 bytes +1,940 AaronSchulz: Deployed b47e9fcfece99 to navtiming
  • curprev 00:52, 21 December 2018imported>Stashbot 127,012 bytes +7,689 twentyafterfour: SWAT Finished. See you all next year!
  • curprev 00:09, 20 December 2018imported>Stashbot 119,323 bytes +8,184 maxsem@deploy1001: Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/479911/ (duration: 00m 53s)
  • curprev 01:27, 19 December 2018imported>Stashbot 111,139 bytes +5,652 tstarling@deploy1001: Synchronized php-1.33.0-wmf.8/extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php: g 480681 make maintenance script dry run more useful (duration: 00m 52s)
  • curprev 00:13, 18 December 2018imported>Stashbot 105,487 bytes +12,885 tgr@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable MediaViewer thumbnail URL guessing for private wikis (T212099) (duration: 00m 45s)
  • curprev 09:52, 16 December 2018imported>Stashbot 92,602 bytes +240 elukey: mask + reset-failed kafkatee default instance on sulfur (kafkatee-webrequest works fine)
  • curprev 23:54, 15 December 2018imported>Nhatminh01m 92,362 bytes −142,693
  • curprev 23:59, 14 December 2018imported>Stashbot 235,055 bytes +4,222 mutante: LDAP: added aezell to wmf group (T211945) for grafana access
  • curprev 01:39, 14 December 2018imported>Stashbot 230,833 bytes +14,006 mutante: install2002 deleted /srv/ contents,then mounted /mnt/vdb on /srv so same content but now / is used only 7% and /srv 57% (T211850)
  • curprev 00:52, 13 December 2018imported>Stashbot 216,827 bytes +9,257 reedy@deploy1001: Synchronized php-1.33.0-wmf.8/includes/http/GuzzleHttpRequest.php: T211806 (duration: 00m 51s)
  • curprev 00:47, 12 December 2018imported>Stashbot 207,570 bytes +10,428 tgr@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: Add some missing groups to the privileged list (duration: 00m 51s)
  • curprev 01:03, 11 December 2018imported>Stashbot 197,142 bytes +9,075 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Configure localized logos for nywiki (T211570) (duration: 01m 36s)
  • curprev 00:55, 10 December 2018imported>Stashbot 188,067 bytes +620 krinkle@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Ic07ff9acfbe17 - T211529, T205546 (duration: 00m 47s)
  • curprev 23:07, 8 December 2018imported>Stashbot 187,447 bytes +460 krinkle@deploy1001: Synchronized docroot/wikipedia.org/speed-tests/: T185446 - I6cf29d598a11 (duration: 00m 47s)
  • curprev 23:54, 7 December 2018imported>Stashbot 186,987 bytes +4,034 ejegg: updated payments-wiki from b99cd0816e to b8acb95a2a
  • curprev 00:47, 7 December 2018imported>Stashbot 182,953 bytes +5,075 XioNoX: done troubleshoting bird bfd on dns2001/cr1-codfw
  • curprev 00:55, 6 December 2018imported>Stashbot 177,878 bytes +11,714 tstarling@deploy1001: Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 47s)
  • curprev 00:32, 5 December 2018imported>Stashbot 166,164 bytes +12,243 dcausse: Evening SWAT done
  • curprev 01:22, 4 December 2018imported>Stashbot 153,921 bytes +10,468 foks: Removing 2FA per request at https://phabricator.wikimedia.org/T210703
  • curprev 00:34, 3 December 2018imported>Stashbot 143,453 bytes +402 legoktm@deploy1001: Synchronized php-1.33.0-wmf.6/includes/Title.php: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/477182 (duration: 00m 52s)
  • curprev 16:48, 1 December 2018imported>Stashbot 143,051 bytes +106 andrewbogott: rebuilding labvirt1014 as cloudvirt1014, T210904
  • curprev 20:23, 30 November 2018imported>Stashbot 142,945 bytes +3,327 ottomata: temporarily disabled puppet on stat1005 to test rsyncd changes
  • curprev 00:57, 30 November 2018imported>Stashbot 139,618 bytes +7,634 thcipriani@deploy1001: Synchronized php-1.33.0-wmf.6/extensions/VisualEditor: SWAT: Rename configs for tracking block notices on visual editor ve.init.mw.ArticleTarget: Stop when we fail to load metadata T209542 (dur...
  • curprev 00:08, 29 November 2018imported>Stashbot 131,984 bytes +8,825 catrope@deploy1001: Synchronized wmf-config/throttle.php: T210681 (duration: 01m 04s)
  • curprev 01:45, 28 November 2018imported>Stashbot 123,159 bytes +10,054 ebernhardson@deploy1001: Synchronized php-1.33.0-wmf.6/extensions/WikimediaEvents/: wbsearchentities needed extension.json to be deployed as well. Sync the whole directory (duration: 00m 53s)
  • curprev 00:38, 27 November 2018imported>Stashbot 113,105 bytes +8,723 niharika29@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Fix weird caching issue maybe for Ian's patch (duration: 00m 46s)
  • curprev 10:40, 25 November 2018imported>Stashbot 104,382 bytes +90 _joe_: restarting pdfrender on scb1002, flapping since 3:00 UTC
  • curprev 21:15, 23 November 2018imported>Stashbot 104,292 bytes +1,690 bawolff: deploy patch for T210192
  • curprev 21:08, 22 November 2018imported>Stashbot 102,602 bytes +4,709 godog: disable raid handler for ms-be2021 - T208096
  • curprev 23:34, 21 November 2018imported>Stashbot 97,893 bytes +6,623 mutante: rsyncing /home from rutherfordium.eqiad to people1001.eqiad (people.wikimedia.org) T210036
  • curprev 00:48, 21 November 2018imported>Stashbot 91,270 bytes +7,660 sbisson@deploy1001: Synchronized wmf-config/InitialiseSettings.php: gerrit:474967 Disable FlaggedRevs, enable RC patrol and add rights on srwikinews (duration: 00m 47s)
  • curprev 00:55, 20 November 2018imported>Stashbot 83,610 bytes +6,935 catrope@deploy1001: Synchronized static/images/project-logos/: Correct logos for Sindhi Wiktionary (duration: 00m 47s)
  • curprev 17:26, 18 November 2018imported>Stashbot 76,675 bytes +136 andrewbogott: restarting cp1078 from mgmt console
  • curprev 07:02, 17 November 2018imported>Stashbot 76,539 bytes +331 volans|off: 'reset modified attributes' on IcingaUI for db1078 (and mgmt) and all its services
  • curprev 23:13, 16 November 2018imported>Stashbot 76,208 bytes +2,465 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@ee91c41]: Deploy on test wdqs1010 (duration: 00m 23s)
  • curprev 00:55, 16 November 2018imported>Stashbot 73,743 bytes +12,226 mutante: some users reported missing files in home dirs on mwmaint1002, reversed rsyncd/ferm setup and rsynced /home from mwmaint2001 to /root on mwmaint1002, restored individually where requested, rsync is not fully automatic but puppetized with rsync...
  • curprev 00:50, 15 November 2018imported>Stashbot 61,517 bytes +9,591 niharika29@deploy1001: Synchronized wmf-config/InitialiseSettings.php: increase Schema.org page split test to 5% sampling T208755 (duration: 00m 54s)
  • curprev 01:02, 14 November 2018imported>Stashbot 51,926 bytes +7,362 thcipriani@deploy1001: Synchronized php-1.33.0-wmf.3/extensions/Wikibase/client/includes: SWAT: Update: use wikibase-debug logger instead of "PageRandomLookup" T208796 (duration: 00m 56s)
  • curprev 19:22, 12 November 2018imported>Stashbot 44,564 bytes +4,905 bawolff@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT T208663 4ff32d1df - Enable moving files for users with patrol and rollbacker rights on srwiki (duration: 00m 54s)
  • curprev 01:10, 10 November 2018imported>Stashbot 39,659 bytes +4,598 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@7eeede7]: redeploy runUpdate.sh for better reporting (duration: 00m 39s)
  • curprev 00:07, 9 November 2018imported>Stashbot 35,061 bytes +5,634 ejegg: updated fundraising CiviCRM from 07183ed7cc to 02cc1f80d4
  • curprev 00:43, 8 November 2018imported>Stashbot 29,427 bytes +10,411 jforrester@deploy1001: Synchronized php-1.33.0-wmf.3/includes/resourceloader/ResourceLoader.php: ResourceLoader: Fail less hard when JSON serialization of config fails I673f59d93 (duration: 00m 53s)
  • curprev 00:46, 7 November 2018imported>Stashbot 19,016 bytes +4,891 mutante: tegmen - shutting down for renaming and reinstall (T208824)
  • curprev 00:26, 6 November 2018imported>Stashbot 14,125 bytes +6,053 maxsem@deploy1001: Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/471875/ (duration: 00m 51s)
  • curprev 23:42, 4 November 2018imported>Stashbot 8,072 bytes +1,014 jynus_: deleting the same row on all s8 broken servers
  • curprev 09:35, 3 November 2018imported>Stashbot 7,058 bytes +112 elukey: run tcpdump on mc1035 to grab memcache traffic (rotating pcaps, ~30G maximum)
  • curprev 17:04, 2 November 2018imported>Stashbot 6,946 bytes +1,465 thcipriani: rollback group2 wikis to 1.33.0-wmf.1 on mwdebug100{1,2}
  • curprev 23:01, 1 November 2018imported>Stashbot 5,481 bytes −179,656 shdubsh: restart hhvm on mw1261
  • curprev 00:58, 1 November 2018imported>Stashbot 185,137 bytes +7,427 onimisionipe: repooling wdqs1004. It has caught up on lag with others
  • curprev 23:23, 30 October 2018imported>Stashbot 177,710 bytes +6,325 thcipriani@deploy1001: Synchronized wmf-config/throttle.php: SWAT: Throttle lift for Wikidata event at University of Edinburgh T208236 (duration: 00m 54s)
  • curprev 21:44, 29 October 2018imported>Stashbot 171,385 bytes +15,633 mutante: contint2001 - installing libtiff upgrade, with the same command from apt/history.log that had shown as failed
  • curprev 23:36, 28 October 2018imported>Stashbot 155,752 bytes +600 krinkle@deploy1001: Synchronized php-1.33.0-wmf.1/extensions/Graph: T184128 - I02da92de33 (duration: 00m 58s)
  • curprev 00:03, 28 October 2018imported>Stashbot 155,152 bytes +2,187 krinkle@deploy1001: Synchronized php-1.33.0-wmf.1/extensions/CirrusSearch: T206967 - Ia23d19cf1e6 (duration: 01m 02s)
  • curprev 00:00, 27 October 2018imported>Stashbot 152,965 bytes +4,090 mutante: icinga1001 - using wmf-auto-reimage to reinstall gets stuck at initial puppet run after reboot - Still waiting for Puppet after 105.0 minutes - aborting on cumin, loggin in directly and manually running puppet (T202782 T208100)
  • curprev 01:41, 26 October 2018imported>Stashbot 148,875 bytes +9,098 bawolff@deploy1001: Synchronized wmf-config/InitialiseSettings.php: T207900 (b74911f6201) enable csp users with session all group1 wikis (duration: 00m 55s)
  • curprev 23:24, 24 October 2018imported>Stashbot 139,777 bytes +6,282 maxsem@deploy1001: Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/469495/ (duration: 00m 54s)
  • curprev 01:46, 24 October 2018imported>Stashbot 133,495 bytes +5,751 tstarling@deploy1001: Synchronized php-1.32.0-wmf.26/includes/page/WikiPage.php: fix deletion performance regression T207530 (duration: 00m 55s)
  • curprev 00:35, 23 October 2018imported>Stashbot 127,744 bytes +10,465 SMalyshev: temp depooled wdq1003 to let it catch up
  • curprev 22:15, 21 October 2018imported>Stashbot 117,279 bytes +401 onimisionipe: repooling wdqs1003 as it has caught up on lag
  • curprev 23:05, 20 October 2018imported>Stashbot 116,878 bytes +662 reedy@deploy1001: Synchronized php-1.32.0-wmf.26/extensions/CentralAuth/: Update setEmail (duration: 00m 55s)
  • curprev 20:33, 19 October 2018imported>Stashbot 116,216 bytes +3,360 twentyafterfour: deployed RCFilters: Fix completely broken highlight circles refs T207472
  • curprev 22:00, 18 October 2018imported>Stashbot 112,856 bytes +11,112 mutante: lvs1011,lvs1012 - manually editing nagios NRPE config and restarting service (to make monitoring from icinga1001 work and puppet is disabled)
  • curprev 00:32, 18 October 2018imported>Stashbot 101,744 bytes +7,362 twentyafterfour: restarting apache on phab1001 to apply b3bfff1138d1212b318392b5e18ac0bfd6f78108
  • curprev 22:11, 16 October 2018imported>Stashbot 94,382 bytes +10,318 ppchelko@deploy1001: Finished deploy [restbase/deploy@d9e3a09]: Downgrade major-greater to minor-greater if no-cache is required, take 6 (duration: 01m 18s)
  • curprev 00:23, 16 October 2018imported>Stashbot 84,064 bytes +10,295 smalyshev@deploy1001: Finished deploy [wdqs/wdqs@c81dd9e]: Redeploy Updater for removal of props channel (duration: 10m 21s)
  • curprev 00:45, 15 October 2018imported>Stashbot 73,769 bytes +1,095 krinkle@deploy1001: Synchronized multiversion/MWRealm.php: I79fb3d194a58: use env.php (duration: 00m 49s)
  • curprev 00:48, 14 October 2018imported>Stashbot 72,674 bytes +412 krinkle@deploy1001: Synchronized php-1.32.0-wmf.24/extensions/CentralAuth/includes/specials/SpecialGlobalGroupMembership.php: T203767 - If2bfa092b (duration: 00m 50s)
  • curprev 18:56, 12 October 2018imported>Stashbot 72,262 bytes +1,505 brion: restarted vp9 background transcodes in eqiad, via mwmaint1002
  • curprev 23:33, 11 October 2018imported>Stashbot 70,757 bytes +8,125 Reedy: ran mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=gomwiki T206741
  • curprev 23:08, 10 October 2018imported>Stashbot 62,632 bytes +8,966 krinkle@deploy1001: Synchronized php-1.32.0-wmf.24/maintenance/resources/foreign-resources.yaml: Ic865e7077d (duration: 00m 49s)
  • curprev 00:19, 10 October 2018imported>Stashbot 53,666 bytes +6,996 krinkle@deploy1001: Synchronized php-1.32.0-wmf.24/includes/utils/UIDGenerator.php: T94522 - I2a0c51bea58 (duration: 00m 56s)
  • curprev 00:04, 9 October 2018imported>Stashbot 46,670 bytes +5,648 krinkle@deploy1001: Synchronized php-1.32.0-wmf.24/includes/libs/rdbms/database: T201900 - I8ae754a2518 (duration: 00m 59s)
  • curprev 16:40, 7 October 2018imported>Stashbot 41,022 bytes +552 dereckson: Reset user email for account "Dominic Mayers" (T206421)
  • curprev 21:20, 6 October 2018imported>Stashbot 40,470 bytes +726 gehel: repooling wdqs2003: catched up on updater lag
  • curprev 23:50, 5 October 2018imported>Stashbot 39,744 bytes +4,624 bblack: <<<<<<< repooling eqiad edge caches, a few days ahead of intended switchback next Weds, to alleviate some traffic engineering concerns over the weekend >>>>>>
  • curprev 00:31, 5 October 2018imported>Stashbot 35,120 bytes +8,957 mutante: LDAP: added user skvjold to group wmf (T204377)
  • curprev 23:54, 3 October 2018imported>Stashbot 26,163 bytes +8,522 mutante: scheduled downtime for wdqs as it's flapping and already known
  • curprev 00:42, 3 October 2018imported>Stashbot 17,641 bytes +3,893 Amir1: Evening SWAT is done
  • curprev 16:53, 2 October 2018imported>Stashbot 13,748 bytes −577,114 jynus: setup test s3 replication channel on db1110 (filtered)
  • curprev 23:21, 18 September 2018imported>Stashbot 590,862 bytes −591,903 twentyafterfour@deploy1001: Synchronized wmf-config/: SWAT: tested on mwdebug1001, now syncing wmf-config settings to the whole cluster (duration: 00m 59s)
  • curprev 00:13, 17 March 2018imported>Stashbot 1,182,765 bytes +4,645 mutante: running puppet on all cache::misc to rename director bromine to webserver_misc_static (T188163)
  • curprev 01:46, 16 March 2018imported>Stashbot 1,178,120 bytes +153 XioNoX: librenms IRC bot moved to -operations channel. Doc on how to turn it off is on https://wikitech.wikimedia.org/wiki/LibreNMS#IRC_Alerting
  • curprev 01:00, 16 March 2018imported>Stashbot 1,177,967 bytes +8,937 reedy@tin: Synchronized php-1.31.0-wmf.25/includes/specials/pagers/NewFilesPager.php: Fix T189846 (duration: 00m 58s)
  • curprev 00:25, 15 March 2018imported>Stashbot 1,169,030 bytes +11,582 tgr@tin: Synchronized php-1.31.0-wmf.24/extensions/Wikibase/client/includes/RecentChanges/ExternalChangeFactory.php: T189320 Use only local part of username when building the RC line (duration: 01m 18s)
  • curprev 00:01, 14 March 2018imported>Stashbot 1,157,448 bytes +6,839 reedy@tin: Synchronized wmf-config/InitialiseSettings.php: crwiki logo (duration: 01m 15s)
  • curprev 22:52, 12 March 2018imported>Stashbot 1,150,609 bytes +7,389 eileen: update civicrm revision changed from a819d64d98 to 8652db05f5, config revision is 08b7e6216e - update civicrm.settings.php
  • curprev 08:50, 11 March 2018imported>Stashbot 1,143,220 bytes +139 elukey: executed sudo rm /etc/logrotate.d/kafkatee-webrequest-analytics on oxygen/rhenium to stop daily cronspam
  • curprev 14:56, 10 March 2018imported>Stashbot 1,143,081 bytes +105 ema: cp1053: restart varnish-be
  • curprev 23:29, 9 March 2018imported>Stashbot 1,142,976 bytes +206 tgr@tin: Synchronized php-1.31.0-wmf.24/extensions/ReadingLists/src/Api/ApiQueryReadingListEntries.php: T189272 fix stupid ReadingLists typo breaking production (duration: 00m 54s)
  • curprev 19:43, 9 March 2018imported>Stashbot 1,142,770 bytes +4,654 foks: changed global email for User:Mathmensch
  • curprev 00:41, 9 March 2018imported>Stashbot 1,138,116 bytes +10,852 thcipriani@tin: Synchronized static/images/project-logos: SWAT: Update logos for Banyumasan and Urdu Wikipedias T189155 PART IV (duration: 00m 58s)
  • curprev 01:17, 8 March 2018imported>Stashbot 1,127,264 bytes +9,869 twentyafterfour: phabricator update completed
  • curprev 00:58, 7 March 2018imported>Stashbot 1,117,395 bytes +10,404 Amir1: Evening SWAT is done
  • curprev 01:00, 6 March 2018imported>Stashbot 1,106,991 bytes +9,749 tgr@tin: Synchronized wmf-config/InitialiseSettings.php: refresh wmf-config/InitialiseSettings, seems to have stuck in old state on some servers after doing the initial sync in the wrong order (duration: 00m 57s)
  • curprev 20:16, 4 March 2018imported>Stashbot 1,097,242 bytes +497 tgr: T188721 ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=wikidatawiki --ignorestatus --logwiki=metawiki 'Erik Fastman' 'Glorious Engine'
  • curprev 14:17, 3 March 2018imported>Stashbot 1,096,745 bytes +725 akosiaris: 13:56:20 ema: powercycle ganeti1005 T181121
  • curprev 01:21, 3 March 2018imported>Stashbot 1,096,020 bytes +3,349 chasemp: labnodepool1001:~# service nodepool stop
  • curprev 00:49, 2 March 2018imported>Stashbot 1,092,671 bytes +12,320 ebernhardson@tin: Synchronized wmf-config/flaggedrevs.php: SWAT: T148603: (duration: 00m 57s)
  • curprev 00:43, 1 March 2018imported>Stashbot 1,080,351 bytes +10,690 catrope@tin: Synchronized wmf-config/InitialiseSettings.php: Clean up $wgEchoPerUserBlacklist setting (duration: 01m 14s)
  • curprev 00:55, 28 February 2018imported>Stashbot 1,069,661 bytes +10,577 demon@tin: Synchronized scap/plugins/wmfbetaautoupdate.py: no-op (duration: 01m 14s)
  • curprev 00:55, 27 February 2018imported>Stashbot 1,059,084 bytes +7,771 catrope@tin: Synchronized wmf-config/InitialiseSettings.php: add very likely bad faith filter on svwiki (T174560) (duration: 00m 57s)
  • curprev 07:36, 25 February 2018imported>Stashbot 1,051,313 bytes +98 marostegui: Fix s7 replication on labsdb1010 - T186579
  • curprev 06:12, 24 February 2018imported>Stashbot 1,051,215 bytes +148 marostegui: Reload haproxy on dbproxy1005
  • curprev 01:18, 24 February 2018imported>Stashbot 1,051,067 bytes +5,059 demon@tin: Synchronized docroot/noc/conf/index.php: fix dblist links from listing (duration: 00m 56s)
  • curprev 01:20, 23 February 2018imported>Stashbot 1,046,008 bytes +9,207 eileen: update civicrm revision changed from aa251f1a93 to a47eafcbad, config revision is c1787646bc
  • curprev 01:05, 22 February 2018imported>Stashbot 1,036,801 bytes +11,188 anomie: Running cleanupBlocks.php on more wikis for T187834: alswiki bgwiki bhwiki cawiki dewiki elwiki eswiki frwiki hewiki hiwiki huwiki hywiki jawiki jawikibooks jawikinews jawikiquote jawikisource jawiktionary kawiki kowiki mswiki mswiktionary rowi...
  • curprev 01:16, 21 February 2018imported>Stashbot 1,025,613 bytes +11,639 eileen: update civicrm revision changed from efba904b06 to 43a7641597, config revision is ef884a2c5d
  • curprev 23:22, 19 February 2018imported>Stashbot 1,013,974 bytes +6,048 eileen: re-enable omnirecipient jobs - process-control config revision is 8c7ce87207
  • curprev 15:50, 18 February 2018imported>Stashbot 1,007,926 bytes +118 _joe_: rolling restart (1 at a time, staggered by 2 minutes) of 18 api appservers in equiad
  • curprev 17:33, 17 February 2018imported>Stashbot 1,007,808 bytes +297 twentyafterfour: restarting apache on phab1001 to clear deadlocked workers. refs T182832
  • curprev 21:12, 16 February 2018imported>Stashbot 1,007,511 bytes +5,314 hashar: Upgraded Zuul to https://gerrit.wikimedia.org/r/#/c/411322/3 | T187567
  • curprev 00:17, 16 February 2018imported>Stashbot 1,002,197 bytes +8,436 ebernhardson@tin: Synchronized php-1.31.0-wmf.21/extensions/ProofreadPage/modules/page/ext.proofreadpage.page.edit.js: SWAT: T187454 fix text selection on #wpTextbox1 (duration: 00m 58s)
  • curprev 01:27, 15 February 2018imported>Stashbot 993,761 bytes +11,159 twentyafterfour: restarting apache2 on phab1001 to free deadlocked php processes.
  • curprev 00:26, 14 February 2018imported>Stashbot 982,602 bytes +9,068 thcipriani@tin: Synchronized wmf-config/InitialiseSettings.php: SWAT: Add uploader user group to mznwiki and make it automagically added T187187 (duration: 01m 12s)
  • curprev 00:24, 13 February 2018imported>Stashbot 973,534 bytes +12,552 cwd: re-enabled p-c
  • curprev 14:06, 11 February 2018imported>Stashbot 960,982 bytes +76 moritzm: installing exim4 security updates on MXs
  • curprev 16:51, 10 February 2018imported>Stashbot 960,906 bytes +187 legoktm@tin: Synchronized php-1.31.0-wmf.20/includes/specials/SpecialLog.php: SpecialLog: Fix results when no offender is specified - T186950 (duration: 00m 57s)
  • curprev 01:11, 10 February 2018imported>Stashbot 960,719 bytes +8,233 demon@tin: Finished deploy [gerrit/gerrit@5d5193e]: one last gitiles rebuild before the weekend (duration: 00m 10s)
  • curprev 00:18, 9 February 2018imported>Stashbot 952,486 bytes +13,814 demon@tin: rebuilt and synchronized wikiversions files: surprise, it broke. revert group1 back to wmf.20
  • curprev 01:29, 8 February 2018imported>Stashbot 938,672 bytes +10,937 andrew@tin: Started deploy [horizon/deploy@9223ba7]: Now with static content, I hope
  • curprev 01:14, 7 February 2018imported>Stashbot 927,735 bytes +10,242 ladsgroup@tin: Synchronized wmf-config/InitialiseSettings.php: Enable fine grained usage tracking, another batch. (T186645) (duration: 01m 11s)
  • curprev 00:25, 6 February 2018imported>Stashbot 917,493 bytes +8,187 thcipriani@tin: Synchronized static/images/mobile/copyright/wikipedia-wordmark-ps.svg: SWAT: Update the ps mobile wordmark T184442 (duration: 00m 55s)
  • curprev 22:41, 4 February 2018imported>Stashbot 909,306 bytes +316 elukey: restart aphlict.service on phab1001 to force it to pick up the new logfile (/var/log/aphlict/aphlict.log rather than the .log.1)
  • curprev 03:55, 3 February 2018imported>Stashbot 908,990 bytes +83 legoktm: restarting zuul to drop 407165,3 from the queue
  • curprev 23:40, 2 February 2018imported>Stashbot 908,907 bytes +4,624 no_justification: gerrit: one last restart to try and force gerrit/phab session restart
  • curprev 23:37, 1 February 2018imported>Stashbot 904,283 bytes +2,967 mutante: creating new 100GB virtual disk for ganeti VM meitnerium (T186020)
  • curprev 23:56, 31 January 2018imported>Stashbot 901,316 bytes +4,831 mutante: restarting apache on phabricator server, same pattern as described in T182832
  • curprev 00:55, 31 January 2018imported>Stashbot 896,485 bytes +1,226 krinkle@tin: Synchronized wmf-config: no-op, adding files for beta cluster (duration: 00m 59s)
  • curprev 00:32, 30 January 2018imported>Stashbot 895,259 bytes +3,803 demon@tin: rebuilt and synchronized wikiversions files: not changing versions, testing something
  • curprev 18:40, 28 January 2018imported>Stashbot 891,456 bytes +41 bblack: testme
  • curprev 16:32, 26 January 2018imported>Stashbot 891,415 bytes +737 addshore@tin: Synchronized wmf-config/InitialiseSettings-labs.php: BETA: Enable FileImporter on testwiki with open config PT 2/2 (duration: 00m 56s)
  • curprev 22:24, 25 January 2018imported>Stashbot 890,678 bytes +783 mutante: restarting gerrit service to apply a few small config changes https://gerrit.wikimedia.org/r/#/q/topic:gerrit-trivial-tweaks+(status:open+OR+status:merged)
  • curprev 00:25, 25 January 2018imported>Stashbot 889,895 bytes +645 urandom: bootstrapping restbase2008-a - T184100
  • curprev 01:15, 24 January 2018imported>Stashbot 889,250 bytes +2,455 matt_flaschen: SWAT complete
  • curprev 22:38, 22 January 2018imported>Stashbot 886,795 bytes +5,025 mutante: rebooting the-server-formerly-known-as-amslvs4 to PXE to reinstall it as bast3003. doesnt work
  • curprev 17:21, 21 January 2018imported>Stashbot 881,770 bytes +246 marostegui: Compress frwiki and jawiki on db1102 - T182450
  • curprev 23:20, 20 January 2018imported>Stashbot 881,524 bytes +421 urandom: bootstrapping restbase2012-a - T184100
  • curprev 22:53, 19 January 2018imported>Stashbot 881,103 bytes +4,027 matt_flaschen: Ran (time foreachwikiindblist flow.dblist extensions/Flow/maintenance/FlowFixInconsistentBoards.php --force) 2>&1|tee --append ~/FlowFixInconsistentBoards_all_2018-01-19_actual_force.txt
  • curprev 01:20, 19 January 2018imported>Stashbot 877,076 bytes +12,450 TimStarling: attempting to restore home_pmtpa from bacula to bast1001
  • curprev 01:08, 18 January 2018imported>Stashbot 864,626 bytes +8,658 twentyafterfour: phabricator deployment finished without incident.
  • curprev 00:29, 17 January 2018imported>Stashbot 855,968 bytes +11,542 ebernhardson@tin: Synchronized wmf-config/InitialiseSettings.php: SWAT: T182616 Remove cirrus AB test config for hewiki (duration: 01m 09s)
  • curprev 23:40, 15 January 2018imported>Stashbot 844,426 bytes +6,099 demon@tin: Synchronized wmf-config/InitialiseSettings.php: turn educationprogram back on for cs.wikipedia -- turns out there was no consensus and a patch should never have been written 😡 (duration: 01m 13s)
  • curprev 20:07, 12 January 2018imported>Stashbot 838,327 bytes +6,725 mutante: mw1227 hhvm-restart
  • curprev 00:51, 12 January 2018imported>Stashbot 831,602 bytes +10,398 thcipriani@tin: Synchronized php-1.31.0-wmf.15/extensions/WikibaseQualityConstraints/extension.json: SWAT: Declare dependency on jquery.makeCollapsible (duration: 01m 21s)
  • curprev 00:57, 11 January 2018imported>Stashbot 821,204 bytes +8,924 urandom: bootstrapping restbase1011-c -- T184100
  • curprev 00:43, 10 January 2018imported>Stashbot 812,280 bytes +10,366 mutante: rebooting gerrit server for kernel upgrade
  • curprev 00:43, 9 January 2018imported>Stashbot 801,914 bytes +9,785 mutante: phabricator servers: upgraded php5-*, openssh
  • curprev 20:25, 7 January 2018imported>Stashbot 792,129 bytes +609 demon@tin: Synchronized wmf-config/interwiki.php: auto-sync with my plugin was busted 🙃 (duration: 00m 50s)
  • curprev 08:09, 6 January 2018imported>Stashbot 791,520 bytes +369 elukey: re-enable eventlogging mysql consumers after database maintenance - T168414
  • curprev 22:28, 5 January 2018imported>Stashbot 791,151 bytes +4,738 tgr: T184263 ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=eswiki --logwiki=metawiki "Mega849" "Mega809"
  • curprev 23:30, 4 January 2018imported>Stashbot 786,413 bytes +6,493 apergos: rebooted releases1001 and 2001 (new kernel)
  • curprev 23:02, 3 January 2018imported>Stashbot 779,920 bytes +9,314 twentyafterfour: restarted apache on phab1001 to clear hung workers (refs T182832)
  • curprev 01:26, 3 January 2018imported>Stashbot 770,606 bytes +9,111 eileen: civicrm updated civicrm revision changed from ffa9d7fc7a to 429a5c5385, config revision is a7b9b58595
  • curprev 22:45, 1 January 2018imported>Stashbot 761,495 bytes +701 eileen1: re-enable silverpop jobs process-control config revision is 4a4c61ae1b
  • curprev 18:22, 30 December 2017imported>Stashbot 760,794 bytes +76 ejegg: disabled failing silverpop data fetch jobs
  • curprev 18:32, 29 December 2017imported>Stashbot 760,718 bytes +813 mutante: gerrit: restart service to apply change from Dec 20th, avoid logspam due to unapplied config change
  • curprev 14:19, 28 December 2017imported>Stashbot 759,905 bytes +71 marostegui: Power cycle pc2005 as it is down
  • curprev 19:54, 27 December 2017imported>Stashbot 759,834 bytes +291 godog: power reset ms-be1033
  • curprev 16:33, 26 December 2017imported>Stashbot 759,543 bytes +1,456 mobrovac@tin: Finished deploy [mathoid/deploy@63b2ddc]: Bring back Mathoid in codfw to v0.6.5 in sync with eqiad - T179419 T172767 (duration: 01m 44s)
  • curprev 16:10, 24 December 2017imported>Stashbot 758,087 bytes +66 _joe_: restarted pdfrenderer on scb1002
  • curprev 12:30, 23 December 2017imported>Stashbot 758,021 bytes +427 addshore@tin: Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY T183500 Re Enable AdvancedSearch on deployment.wikimedia.beta (duration: 00m 51s)
  • curprev 18:31, 22 December 2017imported>Stashbot 757,594 bytes +1,768 mobrovac@tin: Finished deploy [mathoid/deploy@7c5f8e2]: Better handling of chem rules (codfw only) - T183557 (duration: 02m 45s)
  • curprev 19:25, 21 December 2017imported>Stashbot 755,826 bytes +3,470 robh: cp4032 going offline for memory swap
  • curprev 01:17, 21 December 2017imported>Stashbot 752,356 bytes +5,916 eileen: update CiviCRM civicrm revision changed from 9d6ba74f57 to ffa9d7fc7a, config revision is da6fa4cce9
  • curprev 21:51, 19 December 2017imported>Stashbot 746,440 bytes +5,951 XioNoX: removing local-as AS43821 from ams transits - T167840
  • curprev 00:44, 19 December 2017imported>Stashbot 740,489 bytes +6,658 mutante: einsteinium: sudo systemctl restrart ircecho (alias kick-icinga-wm)
  • curprev 01:17, 18 December 2017imported>Stashbot 733,831 bytes +160 cwd: disabled fredge multiqueue consumer
  • curprev 17:27, 16 December 2017imported>Stashbot 733,671 bytes +1,212 no_justification: gerrit: Back, might see a few transient puppet failures if git pulls happened during the d/t, but should all recover
  • curprev 01:16, 16 December 2017imported>Stashbot 732,459 bytes +4,280 mutante: reprepro removing scap 3.7.4-2 package, attempting to reimport 3.7.4-1 package
  • curprev 01:23, 15 December 2017imported>Stashbot 728,179 bytes +9,669 catrope@tin: Synchronized php-1.31.0-wmf.12/extensions/ORES: Fix broken join conditions (T182936) (duration: 00m 57s)
  • curprev 00:12, 14 December 2017imported>Stashbot 718,510 bytes +7,734 catrope@tin: Synchronized wmf-config/InitialiseSettings.php: T182534 (duration: 01m 08s)
  • curprev 00:49, 13 December 2017imported>Stashbot 710,776 bytes +14,879 ebernhardson@tin: Synchronized php-1.31.0-wmf.11/extensions/WikimediaEvents/: SWAT: T182616: turn on second mlr ab test for hewiki (duration: 01m 08s)
  • curprev 00:53, 12 December 2017imported>Stashbot 695,897 bytes +7,791 legoktm@tin: Synchronized wmf-config/CommonSettings.php: Remove manual firejailing of Score binaries (T181535) (duration: 00m 56s)
  • curprev 20:34, 10 December 2017imported>Stashbot 688,106 bytes +317 elukey: execute restart-hhvm on mw1312 - hhvm stuck multiple times queueing requests
  • curprev 17:00, 9 December 2017imported>Stashbot 687,789 bytes +1,492 apergos: restarted hhvm on mw1276, the same old hang with the same old symptoms
  • curprev 00:11, 9 December 2017imported>Stashbot 686,297 bytes +2,589 Jamesofur: removed 2FA from EVinente after verification T182373
  • curprev 00:04, 8 December 2017imported>Stashbot 683,708 bytes +10,510 bblack: cp4026 - restart varnish backend, mailbox lag
  • curprev 23:45, 6 December 2017imported>Stashbot 673,198 bytes +8,263 eileen: update CiviCRM from 85263f625763a454b2664062154c8b791e5a6838 to 2db9c761884c1c48f4e1dd41be416e029b23b481 (improve org merges)
  • curprev 01:01, 6 December 2017imported>Stashbot 664,935 bytes +7,019 eileen: update process-control to 55529ea12d9f16c259fa3 - threshold for $500+ dedupe was too low
  • curprev 00:29, 5 December 2017imported>Stashbot 657,916 bytes +9,704 mutante: bast4001 - removing ganglia aggregators, package, config...
  • curprev 15:33, 3 December 2017imported>Stashbot 648,212 bytes +641 ejegg: disabled CiviCRM bounce processing job
  • curprev 17:56, 2 December 2017imported>Stashbot 647,571 bytes +231 marostegui: Reboot db1096.s5 to pick up the correct innodb_buffer_pool size after finishing compressing s5 - T178359
  • curprev 00:03, 2 December 2017imported>Stashbot 647,340 bytes +5,857 mutante: tried one more time on db2028,db2029, both trusty. on db2028: gmond was running as user ganglia-monitor, failed, had to manually kill the process, run puppet again then ok. on db2029, gmond was running as "499" but puppet just ran and removed...
  • curprev 00:13, 1 December 2017imported>Stashbot 641,483 bytes +8,175 maxsem@tin: Synchronized wmf-config/InitialiseSettings.php: HTML5 sections be upon us! (duration: 00m 45s)
  • curprev 01:17, 30 November 2017imported>Stashbot 633,308 bytes +8,979 ejegg: updated fundraising tools from 6d4b6f388e64ff4dc9a6a501bc8339fc1294fe9f to 6e604fd9f3ca57935e2c35e37b7e85b4ca438d6a
  • curprev 01:25, 29 November 2017imported>Stashbot 624,329 bytes +9,402 mutante: snapshot1001 - closed idle screen session
  • curprev 22:29, 27 November 2017imported>Stashbot 614,927 bytes +10,005 mutante: labtestnet2001 - re-enabled puppet to decom ganglia, no errors
  • curprev 19:31, 25 November 2017imported>Stashbot 604,922 bytes +1,379 marostegui: Set 32:3 disk to offline on db1051
  • curprev 13:52, 24 November 2017imported>Stashbot 603,543 bytes +3,195 moritzm: removing git packages from jessie-wikimedia/experimental (replaced by component/git)
  • curprev 00:12, 24 November 2017imported>Stashbot 600,348 bytes +4,938 reedy@tin: Synchronized docroot/mediawiki/keys/: Add Brian Wolff's key (duration: 00m 45s)
  • curprev 01:29, 23 November 2017imported>Stashbot 595,410 bytes +7,884 demon@tin: Synchronized wmf-config/CommonSettings.php: Set $wgCentralAuthGlobalBlockInterwikiPrefix (duration: 00m 45s)
  • curprev 01:09, 22 November 2017imported>Stashbot 587,526 bytes +10,099 hoo: Cleaned out remaining T180934 related log blow up on snapshot1007 (dumpwikidatajson-wikidata-20171120-all-0.log)
  • curprev 00:19, 21 November 2017imported>Stashbot 577,427 bytes +11,039 catrope@tin: Synchronized php-1.31.0-wmf.8/resources/src/mediawiki.rcfilters/ui/mw.rcfilters.ui.ItemMenuOptionWidget.js: T180863 (duration: 00m 49s)
  • curprev 23:20, 19 November 2017imported>Stashbot 566,388 bytes +480 Jamesofur: removed 2FA for Ask21 T180889
  • curprev 20:06, 18 November 2017imported>Stashbot 565,908 bytes +299 gehel: upgrade of elasticsearch eqiad complete - T178411
  • curprev 00:30, 18 November 2017imported>Stashbot 565,609 bytes +5,982 urandom: Bootstrapping restbase2006-b (T179422)
  • curprev 01:28, 17 November 2017imported>Stashbot 559,627 bytes +11,959 ejegg: updated CiviCRM from 8454e0639eed6d4a0e7b01970dd5a6f9d426cb29 to 272580a4b0511b7eb5392dc087a2d21e56474ab9
  • curprev 01:29, 16 November 2017imported>Stashbot 547,668 bytes +11,330 twentyafterfour: Phabricator will be offline for a couple of minutes while I apply database migrations.
  • curprev 00:59, 15 November 2017imported>Stashbot 536,338 bytes +9,146 ejegg: updated payments-wiki from d150287f0b64d13589e1af728be0cb5b89f66699 to 210cb37d9973eabed06a520ac31fe74d30ffde4a
  • curprev 22:49, 13 November 2017imported>Stashbot 527,192 bytes +7,588 bawolff: deployed patch T119158 (Will affect language converter. -{}- no longer allowed in link urls)
  • curprev 19:39, 12 November 2017imported>Stashbot 519,604 bytes +153 urandom: Decommissioning Cassandra, restbase1007-b.eqiad.wmnet (T179422)
  • curprev 01:29, 12 November 2017imported>Stashbot 519,451 bytes +214 urandom: Decommissioning Cassandra, restbase1007-a.eqiad.wmnet (T179422)
  • curprev 00:48, 11 November 2017imported>Stashbot 519,237 bytes +4,117 urandom: Decommissioning Cassandra, restbase2006-b.codfw.wmnet (T179422)
  • curprev 01:21, 10 November 2017imported>Stashbot 515,120 bytes +4,946 aaron@tin: Synchronized php-1.31.0-wmf.7/includes/db: Use the main stash for LBFactory "memStash" parameter (duration: 00m 47s)
  • curprev 01:02, 9 November 2017imported>Stashbot 510,174 bytes +12,205 twentyafterfour: Not deploying any phabricator updates this week.
  • curprev 23:58, 7 November 2017imported>Stashbot 497,969 bytes +12,841 urandom: Decommissioning restbase2001-b.codfw.wmnet (T179422)
  • curprev 22:34, 6 November 2017imported>Stashbot 485,128 bytes +10,962 bawolff: Deploy patch for T178451
  • curprev 13:34, 4 November 2017imported>Stashbot 474,166 bytes +274 elukey@puppetmaster1001: conftool action : set/pooled=yes; selector: name=scb1002.eqiad.wmnet
  • curprev 00:18, 4 November 2017imported>Stashbot 473,892 bytes +3,306 Krinkle: Finished whisper-mass-resize for frontend.navtiming on graphite2001 (T179622)
  • curprev 23:36, 2 November 2017imported>Stashbot 470,586 bytes +6,195 krinkle@tin: Synchronized php-1.31.0-wmf.6/extensions/NavigationTiming/modules/ext.navigationTiming.js: Fix zero values - T178479 (duration: 00m 47s)
  • curprev 00:52, 2 November 2017imported>Stashbot 464,391 bytes +7,145 ebernhardson@tin: Synchronized php-1.31.0-wmf.5/extensions/WikimediaEvents/extension.json: SWAT followup: Update SearchSatisfaction eventlogging schema revision id (duration: 00m 50s)
  • curprev 23:56, 31 October 2017imported>Stashbot 457,246 bytes +7,141 ebernhardson@tin: Synchronized wmf-config/InitialiseSettings.php: Adjust cirrussearch interleave configuration for AB test (duration: 00m 50s)
  • curprev 23:28, 30 October 2017imported>Stashbot 450,105 bytes +11,992 ebernhardson@tin: Synchronized wmf-config/InitialiseSettings.php: Update cirrussearch MLR model for enwiki (duration: 00m 50s)
  • curprev 23:49, 29 October 2017imported>Stashbot 438,113 bytes +757 ema: powercycle cp4024
  • curprev 21:03, 28 October 2017imported>Stashbot 437,356 bytes +1,408 bblack: cp1067 (current target cache): disabling the relatively-new VCL that sets do_stream=false if !CL on applayer fetches...
  • curprev 20:54, 27 October 2017imported>Stashbot 435,948 bytes +2,656 MaxSem: running migratePreferences.php on group2 wikis
  • curprev 22:47, 26 October 2017imported>Stashbot 433,292 bytes +10,731 demon@tin: rebuilt wikiversions.php and synchronized wikiversions files: wikidata to wmf.4
  • curprev 23:48, 25 October 2017imported>Stashbot 422,561 bytes +8,480 legoktm@tin: Synchronized php-1.31.0-wmf.5/extensions/VisualEditor/modules/ve-mw/ui/styles/contextitems/ve.ui.MWInternalLinkContextItem.css: MWInternalLinkContextItem: increase specificity to override OOUI changes - T178933 (duration: 00m 50s)
  • curprev 22:55, 24 October 2017imported>Stashbot 414,081 bytes +5,808 ejegg: updated civicrm from from 0546446640c8b7485c78b4f3836a38630248b44e to 25c7a33e3e3e9b8a5020d826982b3dbb64057614
  • curprev 23:09, 23 October 2017imported>Stashbot 408,273 bytes +7,033 demon@tin: Synchronized wmf-config/throttle.php: Working Class Movement Library (Salford) throttle rule (duration: 00m 46s)
  • curprev 08:16, 21 October 2017imported>Stashbot 401,240 bytes +203 hashar: Mass code-review+2 changes made by LibraryUpdater that already had Code-Review:+2 and NOT verified=-1 ( ping legoktm )
  • curprev 23:46, 20 October 2017imported>Stashbot 401,037 bytes +2,650 demon@tin: Finished deploy [gerrit/gerrit@21b7332]: pushing deleteproject plugin (duration: 00m 09s)
  • curprev 22:21, 19 October 2017imported>Stashbot 398,387 bytes +5,288 addshore: stop running extra change dispatchers on terbium for wikidatawiki
  • curprev 00:01, 19 October 2017imported>Stashbot 393,099 bytes +7,053 maxsem@tin: Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/385106/2 (duration: 00m 50s)
  • curprev 23:31, 17 October 2017imported>Stashbot 386,046 bytes +9,416 thcipriani@tin: Synchronized php-1.31.0-wmf.4/extensions/Citoid/modules/ve.ui.CiteFromIdInspector.css: SWAT: ve.ui.CiteFromIdInspector: Fix CSS for context menus after changes in OOjs UI T178324 (duration: 00m 50s)
  • curprev 23:49, 16 October 2017imported>Stashbot 376,630 bytes +9,132 thcipriani@tin: Synchronized php-1.31.0-wmf.3/extensions/CheckUser/specials/SpecialCheckUser.php: SWAT: Revert "Revert "Restore checkuser-userlinks when exist"" T170507 (duration: 00m 46s)
  • curprev 11:16, 15 October 2017imported>Stashbot 367,498 bytes +134 mobrovac@tin: Started restart [electron-render/deploy@8dd5f13]: Electron hanging - T174916
  • curprev 03:53, 14 October 2017imported>Stashbot 367,364 bytes +148 paravoid: manually cleaning up pdns_metric_cron from all dnsrecursors (hydrogen/chromium/acamar/achernar/maerlant/nescio)
  • curprev 23:04, 13 October 2017imported>Stashbot 367,216 bytes +3,525 mutante: scb1001 - systemctl restart pdfrender (Icinga said: pdfrender Connection refused)
  • curprev 00:23, 13 October 2017imported>Stashbot 363,691 bytes +4,138 mutante: authdns servers: deleted /etc/ganglia/conf.d/gndsd.pyconf & /usr/lib/ganglia/python_modules_gdnsd.py - removing ganglia stats
  • curprev 00:30, 12 October 2017imported>Stashbot 359,553 bytes +13,504 twentyafterfour: phabricator update finished (a while ago, I forgot to log it)
  • curprev 23:54, 10 October 2017imported>Stashbot 346,049 bytes +6,831 thcipriani@tin: Synchronized wmf-config: SWAT: Disable OCG services PART II T177795 (duration: 00m 48s)
  • curprev 01:21, 10 October 2017imported>Stashbot 339,218 bytes +9,320 eileen2: update civicrm from 276cbfd4ebbf4b1630ab9eb87fe5ea365d624767 to 00da4b0e7c03e8ffebbf0de580203d50b67b998e
  • curprev 21:40, 8 October 2017imported>Stashbot 329,898 bytes +351 hoo: Killed "dumpwikidatardf.sh truthy nt" (wikidata truthy dump) on snapshot1007, got stuck after T169680.
  • curprev 22:46, 7 October 2017imported>Stashbot 329,547 bytes +254 reedy@tin: Synchronized php-1.31.0-wmf.2/resources/: T177705 (duration: 00m 48s)
  • curprev 18:58, 6 October 2017imported>Stashbot 329,293 bytes +3,595 ejegg: updated CiviCRM from 197b22dce5a6c60a6e3697f5b9ba49de536d0bbb to 276cbfd4ebbf4b1630ab9eb87fe5ea365d624767
  • curprev 23:40, 5 October 2017imported>Stashbot 325,698 bytes +2,876 ejegg: updated CiviCRM from cca1a9d90ca41fe9fc75c12095225220dcc43226 to 197b22dce5a6c60a6e3697f5b9ba49de536d0bbb
  • curprev 20:11, 5 October 2017imported>Stashbot 322,822 bytes +10,067 twentyafterfour@tin: rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.31.0-wmf.2 refs T174358
  • curprev 00:06, 5 October 2017imported>Stashbot 312,755 bytes +12,305 catrope@tin: Synchronized php-1.31.0-wmf.2/resources/src/mediawiki.rcfilters/mw.rcfilters.init.js: T176652 (duration: 00m 49s)
  • curprev 00:33, 4 October 2017imported>Stashbot 300,450 bytes +6,435 ejegg: re-enabled fundraising queue consumers
  • curprev 23:38, 2 October 2017imported>Stashbot 294,015 bytes +6,935 catrope@tin: Synchronized php-1.31.0-wmf.1/extensions/VisualEditor/: T177250 (duration: 00m 47s)
  • curprev 21:30, 1 October 2017imported>Stashbot 287,080 bytes +214 madhuvishy: powercycling labvirt1015
  • curprev 20:54, 30 September 2017imported>Stashbot 286,866 bytes +149 catrope@tin: Synchronized wmf-config/CommonSettings.php: Bump $wgResourceLoaderStorageVersion (T176884) (duration: 00m 47s)
  • curprev 00:37, 30 September 2017imported>Stashbot 286,717 bytes +2,628 ejegg: increased PayPal Express Checkout cURL timeout to 12 seconds
  • curprev 00:26, 29 September 2017imported>Stashbot 284,089 bytes +8,921 ejegg: updated fundraising tools from 69a96276fa8cd8f68aa1225ea48fff1a7c833101 to 6d4b6f388e64ff4dc9a6a501bc8339fc1294fe9f
  • curprev 00:36, 28 September 2017imported>Stashbot 275,168 bytes +11,590 twentyafterfour: upgrading phabricator, expect minimal downtime
  • curprev 23:25, 26 September 2017imported>Stashbot 263,578 bytes +6,633 thcipriani@tin: Synchronized php-1.31.0-wmf.1/extensions/VisualEditor/modules/ve-mw/ui/tools/ve.ui.MWSignatureTool.js: SWAT: Revert MWSignatureTool case (duration: 00m 48s)
  • curprev 23:58, 25 September 2017imported>Stashbot 256,945 bytes +6,716 ejegg: updated payments-wiki from d2ce6970a761829de86d62919bca8ebd821f41f1 to 6104e3dea7a7051305c7d76f1e913b2410cbfcbf
  • curprev 14:23, 24 September 2017imported>Stashbot 250,229 bytes +201 jynus: restarting db2047 for kernel upgrade
  • curprev 00:18, 24 September 2017imported>Quiddity 250,028 bytes −1,084,967 Archived 2017-01-01 through 2017-07-31 to Server_admin_log/Archive_32
  • curprev 23:36, 22 September 2017imported>Stashbot 1,334,995 bytes +1,342 mutante: gerrit2001 - rebooting .. wave a dead chicken http://www.catb.org/jargon/html/W/wave-a-dead-chicken.html
  • curprev 23:57, 21 September 2017imported>Stashbot 1,333,653 bytes +9,663 ejegg: updated payments-wiki from 7ed72430d771361efe053d60a127827e7c917823 to d2ce6970a761829de86d62919bca8ebd821f41f1
  • curprev 00:42, 21 September 2017imported>Stashbot 1,323,990 bytes +2,571 eileen: update civicrm from da833ac561f7586a0b64fbef2cdf2963bc4055c3 to fc56901efd386ca725bde971ae000d798e1e2fca
  • curprev 21:10, 20 September 2017imported>Stashbot 1,321,419 bytes +43,832 bblack@neodymium: conftool action : set/pooled=no; selector: name=cp4015.ulsfo.wmnet
  • curprev 01:19, 13 September 2017imported>Stashbot 1,277,587 bytes +7,211 jynus: stopping mariadb @ db1048 to clone it to db1059
  • curprev 00:48, 12 September 2017imported>Stashbot 1,270,376 bytes +5,020 awight@tin: Finished deploy [ores/deploy@42c5663]: Cause ORES service restart (duration: 00m 18s)
  • curprev 21:00, 10 September 2017imported>Stashbot 1,265,356 bytes +270 bawolff: disabled Jforrester's gerrit account and flush sessions
  • curprev 00:37, 10 September 2017imported>Stashbot 1,265,086 bytes +11,360 chasemp: labnodepool1001:~# service nodepool restart
  • curprev 01:29, 7 September 2017imported>Stashbot 1,253,726 bytes +8,800 eileen: replag 0
  • curprev 00:15, 6 September 2017imported>Stashbot 1,244,926 bytes +7,161 Reedy: that was T174747 Adjust language icon color
  • curprev 23:32, 4 September 2017imported>Stashbot 1,237,765 bytes +3,941 Dereckson: Set e-mail to Hexabot bot account to allow account recovery (T174973)
  • curprev 17:53, 3 September 2017imported>Stashbot 1,233,824 bytes +540 elukey: depooled cp4024 (ulsfo upload) due to kernel errors in dmesg
  • curprev 01:00, 3 September 2017imported>Stashbot 1,233,284 bytes +200 legoktm: legoktm@terbium:~$ mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --wiki=hiwikiversity --backend=local-multiwrite # T174859
  • curprev 16:18, 1 September 2017imported>Stashbot 1,233,084 bytes +2,262 bblack: restart varnish backend on cp1074 (mailbox lag)
  • curprev 00:57, 1 September 2017imported>Stashbot 1,230,822 bytes +6,348 tstarling@tin: Synchronized php-1.30.0-wmf.16/includes/parser/Parser.php: (no justification provided) (duration: 00m 44s)
(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)