You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Server Admin Log: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

5 November 2020

  • curprev 01:4701:47, 5 November 2020imported>Stashbot 784,617 bytes +19,362 milimetric@deploy1001: Started deploy [analytics/refinery@6913407]: Regular analytics weekly train [analytics/refinery@6913407]

3 November 2020

1 November 2020

  • curprev 22:4122:41, 1 November 2020imported>Stashbot 747,823 bytes +627 Urbanecm: mwscript extensions/OATHAuth/maintenance/disableOATHAuthForUser.php --wiki=metawiki Turkmen # T266976
  • curprev 01:2601:26, 1 November 2020imported>Stashbot 747,196 bytes +356 rzl@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)

31 October 2020

  • curprev 00:1200:12, 31 October 2020imported>Stashbot 746,840 bytes +5,375 mutante: removed Nuria from wmf group, she is already in nda group (T266086)

30 October 2020

29 October 2020

  • curprev 01:4101:41, 29 October 2020imported>Stashbot 726,980 bytes +9,799 mutante: scandium reimaged a second time after making puppet changes to ensure nodejs/npm is NOT installed anymore (T257906)

28 October 2020

  • curprev 00:4000:40, 28 October 2020imported>Stashbot 717,181 bytes +10,760 eileen: civicrm revision changed from 4fdfb8408b to e1d65b0f3a, config revision is f16003ab62

26 October 2020

  • curprev 23:1223:12, 26 October 2020imported>Stashbot 706,421 bytes +13,445 catrope@deploy1001: Synchronized php-1.36.0-wmf.14/extensions/GrowthExperiments/: Fix JS error when no topics set (T266501) (duration: 01m 00s)

25 October 2020

23 October 2020

  • curprev 22:5622:56, 23 October 2020imported>Stashbot 692,840 bytes +1,858 mutante: added Nuria to "nda" LDAP group - leaving her in "wmf" until the actual last day - shell account remains so no puppet change needed in ldap_only_admins (T266086)

22 October 2020

  • curprev 22:4222:42, 22 October 2020imported>Stashbot 690,982 bytes +6,223 mutante: ganeti1001 - adding 2 more vcpus to VM testreduce1001 - T257940

21 October 2020

  • curprev 23:1623:16, 21 October 2020imported>Stashbot 684,759 bytes +8,846 catrope@deploy1001: Synchronized php-1.36.0-wmf.14/extensions/GrowthExperiments/: T266033 (duration: 01m 05s)

20 October 2020

  • curprev 22:1022:10, 20 October 2020imported>Stashbot 675,913 bytes +8,976 dwisehaupt: frmon2001 upgraded to buster with grafana 7.2.1

19 October 2020

  • curprev 23:5723:57, 19 October 2020imported>Stashbot 666,937 bytes +14,227 gehel@cumin1001: END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99)

17 October 2020

  • curprev 13:2213:22, 17 October 2020imported>Stashbot 652,710 bytes +173 Urbanecm: [urbanecm@mwmaint2001 ~/uploads]$ mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Fæ . # T264529

16 October 2020

  • curprev 21:4621:46, 16 October 2020imported>Stashbot 652,537 bytes +3,177 pt1979@cumin2001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)
  • curprev 01:0101:01, 16 October 2020imported>Stashbot 649,360 bytes +9,475 ryankemper: Cleaning up a dangling no-longer-puppet-managed udev elasticsearch-readahead rule across all cirrus instances: `sudo cumin -b 36 C:profile::elasticsearch::cirrus 'sudo rm -fv /etc/udev/rules.d/elasticsearch-readahead.rules && sudo /sbin/udevadm control --reload && sudo /sbin/udevadm trigger'`

15 October 2020

  • curprev 00:3600:36, 15 October 2020imported>Stashbot 639,885 bytes +8,944 ryankemper: Beginning rolling upgrade for cirrussearch `eqiad`. Cookbook will restart elasticsearch on 36 nodes total, 3 nodes at a time

13 October 2020

  • curprev 23:2223:22, 13 October 2020imported>Stashbot 630,941 bytes +8,939 catrope@deploy1001: Synchronized php-1.36.0-wmf.13/extensions/GrowthExperiments/: Revert removal of variant A (T265372) (duration: 01m 04s)

12 October 2020

  • curprev 17:0317:03, 12 October 2020imported>Stashbot 622,002 bytes +1,861 jayme: fixed /var/lock/ permission (1777) on ms-be2036 - T265208

10 October 2020

9 October 2020

  • curprev 00:1600:16, 9 October 2020imported>Stashbot 611,953 bytes +10,880 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0)

8 October 2020

7 October 2020

  • curprev 01:0001:00, 7 October 2020imported>Stashbot 591,948 bytes +13,893 cdanis: repool esams; cr2-esams router upgrade complete

5 October 2020

  • curprev 23:1123:11, 5 October 2020imported>Stashbot 578,055 bytes +8,516 ejegg: updated payments staging from 52704ffe24 to db03677b2d

3 October 2020

  • curprev 15:5215:52, 3 October 2020imported>Stashbot 569,539 bytes +246 urbanecm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: emergency: 840545f1d9115ea6b672cecce1762d850d8b1f54: Restrict flow-hide right to autoconfirmed users on zhwiki (T264489) (duration: 01m 17s)
  • curprev 00:0800:08, 3 October 2020imported>Stashbot 569,293 bytes +9,618 ejegg: updated fundraising CiviCRM from 256adda03c to a30da7f92a

1 October 2020

  • curprev 23:3823:38, 1 October 2020imported>Stashbot 559,675 bytes +14,499 ebernhardson@deploy1001: Finished deploy [wikimedia/discovery/analytics@6101b56]: mjolnir: increase training memory overhead by 10% (duration: 00m 34s)

30 September 2020

29 September 2020

27 September 2020

26 September 2020

24 September 2020

  • curprev 23:3923:39, 24 September 2020imported>Stashbot 493,213 bytes +19,136 andrew@deploy1001: Finished deploy [horizon/deploy@7b61460]: (no justification provided) (duration: 01m 58s)

23 September 2020

  • curprev 23:5223:52, 23 September 2020imported>Stashbot 474,077 bytes +20,309 mutante: alert1001 - systemctl restar ircecho because icinga-wm left the chat

22 September 2020

  • curprev 23:2723:27, 22 September 2020imported>Stashbot 453,768 bytes +20,420 legoktm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: clientError: enable on ja,es,de,ru,it,zh,pt wikipedias (T255585) (duration: 01m 04s)

21 September 2020

  • curprev 23:4223:42, 21 September 2020imported>Stashbot 433,348 bytes +16,298 mholloway-shell@deploy1001: helmfile [codfw] Ran 'sync' command on namespace 'push-notifications' for release 'main' .

20 September 2020

  • curprev 08:4608:46, 20 September 2020imported>Stashbot 417,050 bytes +345 Urbanecm: [urbanecm@mwmaint2001 ~]$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'Tepig10102020' 'Davidfromtheworld' # T263317

19 September 2020

  • curprev 19:0319:03, 19 September 2020imported>Stashbot 416,705 bytes +407 ariel@deploy1001: Finished deploy [dumps/dumps@14ba6e9]: defer getting db creds until really needed (duration: 00m 04s)

18 September 2020

17 September 2020

16 September 2020

  • curprev 23:4123:41, 16 September 2020imported>Stashbot 389,397 bytes +13,843 catrope@deploy1001: Synchronized php-1.36.0-wmf.8/extensions/FlaggedRevs: T262970 (duration: 01m 06s)

15 September 2020

  • curprev 23:2023:20, 15 September 2020imported>Stashbot 375,554 bytes +10,278 urbanecm@deploy1001: Synchronized php-1.36.0-wmf.9/extensions/FlaggedRevs/backend/FlaggedRevsHooks.php: 1c0b0d161fe1024d6d08a27bbacf5b62c56c9c01: Fix APCOND_FR_NEVERBLOCKED handling (T262970) (duration: 00m 56s)

14 September 2020

  • curprev 22:5922:59, 14 September 2020imported>Stashbot 365,276 bytes +15,318 mholloway-shell@deploy1001: helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' .

13 September 2020

  • curprev 23:4723:47, 13 September 2020imported>Stashbot 349,958 bytes +218 Urbanecm: Change email address of User:Oversight@enwiki to oversight-l@lists.wikimedia.org as part of OTRS downtime preparation (T262733)

12 September 2020

11 September 2020

  • curprev 00:4100:41, 11 September 2020imported>Stashbot 345,704 bytes +10,447 milimetric@deploy1001: Finished deploy [analytics/refinery@7f5a6ca] (thin): Regular analytics weekly train THIN [analytics/refinery@7f5a6ca] (duration: 00m 08s)

10 September 2020

  • curprev 00:3100:31, 10 September 2020imported>Stashbot 335,257 bytes +14,554 ebernhardson@deploy1001: Finished deploy [wikimedia/discovery/analytics@00b0e20]: Update to current master (duration: 06m 42s)

8 September 2020

  • curprev 23:4723:47, 8 September 2020imported>Stashbot 320,703 bytes +17,255 eileen: civicrm revision is 4e40a59d42, config revision is d26334fa36

7 September 2020

6 September 2020

  • curprev 19:4519:45, 6 September 2020imported>Stashbot 294,441 bytes +598 marostegui@cumin1001: dbctl commit (dc=all): 'Decrease db2127's weight a bit', diff saved to https://phabricator.wikimedia.org/P12496 and previous config saved to /var/cache/conftool/dbconfig/20200906-194512-marostegui.json

5 September 2020

4 September 2020

  • curprev 01:3501:35, 4 September 2020imported>Stashbot 289,064 bytes +20,066 pt1979@cumin2001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)

2 September 2020

1 September 2020

  • curprev 22:3922:39, 1 September 2020imported>Stashbot 257,665 bytes +5,601 Urbanecm: [urbanecm@mwmaint2001 ~]$ mwscript extensions/OATHAuth/maintenance/disableOATHAuthForUser.php --wiki=sysop_itwiki Pierpao (T261722)
  • curprev 00:0200:02, 1 September 2020imported>Stashbot 252,064 bytes +8,872 mutante: wb2-grrrri was not running and wikibugs had no more Gerrit updates since a while

30 August 2020

29 August 2020

  • curprev 18:0518:05, 29 August 2020imported>Stashbot 243,128 bytes +402 Amir1: end of ladsgroup@mwmaint1002:~$ foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T261451)

28 August 2020

  • curprev 21:5321:53, 28 August 2020imported>Stashbot 242,726 bytes +5,179 ryankemper: `sudo systemctl reload nginx.service` on `cloudelastic100[5,6].wikimedia.org` to try to resolve certificate warning issues
  • curprev 00:0600:06, 28 August 2020imported>Stashbot 237,547 bytes +27,285 eileen: process-control config revision is dd541a25dc

26 August 2020

  • curprev 23:3523:35, 26 August 2020imported>Stashbot 210,262 bytes +10,539 eileen: civicrm revision changed from d2e80f7522 to 481ab742db, config revision is e2ab4d7c1f

25 August 2020

  • curprev 21:5121:51, 25 August 2020imported>Stashbot 199,723 bytes +10,845 mutante: xhgui1001/xhgui2001 - Unpacking xhgui (0.12.0-2-wmf1) over (0.9.0-1-wmf1) (T260397)
  • curprev 00:5700:57, 25 August 2020imported>Stashbot 188,878 bytes +12,712 cdanis: T259621 ❌cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 homer 'cr*' commit 'drain cr2-esams transport link'

23 August 2020

  • curprev 20:3820:38, 23 August 2020imported>Stashbot 176,166 bytes +518 cmjohnson@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)

22 August 2020

  • curprev 19:3319:33, 22 August 2020imported>Stashbot 175,648 bytes +367 ryankemper: depooled wdqs1006 (still has 2.5 hours to catch up on)

21 August 2020

  • curprev 17:4317:43, 21 August 2020imported>Stashbot 175,281 bytes +1,663 cmjohnson@cumin1001: END (PASS) - Cookbook sre.dns.netbox (exit_code=0)

20 August 2020

  • curprev 22:3122:31, 20 August 2020imported>Stashbot 173,618 bytes +10,495 eileen: civicrm revision changed from 27d5900f7d to ce28723709, config revision is 706cf3c898
  • curprev 00:5100:51, 20 August 2020imported>Stashbot 163,123 bytes +12,052 mutante: ms-be1039 - started failed ferm service

19 August 2020

  • curprev 01:0501:05, 19 August 2020imported>Stashbot 151,071 bytes +12,992 dpifke@deploy1001: Synchronized wmf-config/profiler.php: Deploying https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/620139 (duration: 01m 18s)

18 August 2020

  • curprev 00:5300:53, 18 August 2020imported>Stashbot 138,079 bytes +29,239 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0)

16 August 2020

15 August 2020

  • curprev 21:1821:18, 15 August 2020imported>Stashbot 108,768 bytes +123 gehel: depooling wdqs1004 and restarting services, will wait to catch up on lag before repooling

14 August 2020

13 August 2020

  • curprev 23:3923:39, 13 August 2020imported>Stashbot 107,903 bytes +7,860 tzatziki: removing 3 files for legal compliance
  • curprev 00:3700:37, 13 August 2020imported>Stashbot 100,043 bytes +31,681 mutante: removing jenkins_service_running checks from secondary servers where it's stopped, manually from icinga config, running puppet on icinga

11 August 2020

  • curprev 23:4123:41, 11 August 2020imported>Stashbot 68,362 bytes +9,831 Urbanecm: Evening B&C window completed
  • curprev 00:3300:33, 11 August 2020imported>Stashbot 58,531 bytes +11,201 dpifke@deploy1001: Finished deploy [performance/arc-lamp@fc5f1c6]: Deploying latest attempt to fix T259167 (duration: 01m 03s)

9 August 2020

  • curprev 21:5821:58, 9 August 2020imported>Stashbot 47,330 bytes +195 ejegg: updated payments-wiki from cd012f37f1 to 932aacde54

8 August 2020

  • curprev 02:2302:23, 8 August 2020imported>Stashbot 47,135 bytes +243 ryankemper@cumin1001: START - Cookbook sre.wdqs.data-reload

7 August 2020

  • curprev 16:4216:42, 7 August 2020imported>Stashbot 46,892 bytes +2,081 jforrester@deploy1001: Synchronized php-1.36.0-wmf.3/extensions/DiscussionTools/: T259855 Revert new reply API (duration: 01m 06s)

6 August 2020

  • curprev 23:2123:21, 6 August 2020imported>Stashbot 44,811 bytes +12,330 catrope@deploy1001: Synchronized php-1.36.0-wmf.3/extensions/GrowthExperiments/: Fixes for WelcomeSurvey language question (T232410) (duration: 00m 59s)
  • curprev 00:3500:35, 6 August 2020imported>Stashbot 32,481 bytes +9,981 mutante: wtp2019 - reimaging - parsoid service does not work, unlike on all other wtp*, making sure it's clean

4 August 2020

  • curprev 22:4122:41, 4 August 2020imported>Stashbot 22,500 bytes +12,987 brennen: restarting php7.2-fpm on mw1404 for opcache issues

3 August 2020

  • curprev 23:4323:43, 3 August 2020imported>Stashbot 9,513 bytes +9,109 mutante: mwdebug1001 - temp installing apt-file for debugging an issue on mwmaint

1 August 2020

  • curprev 16:3016:30, 1 August 2020imported>Stashbot 404 bytes −726,353 Amir1: wikiadmin@10.64.32.197(avkwiki)> delete from site_identifiers; (T259122)

31 July 2020

  • curprev 23:4823:48, 31 July 2020imported>Stashbot 726,757 bytes +6,358 ejegg: updated payments-wiki from c365c136d2 to cd012f37f1
  • curprev 00:4700:47, 31 July 2020imported>Stashbot 720,399 bytes +10,664 tstarling@deploy1001: Synchronized wmf-config/CommonSettings.php: disable lilypond execution again (duration: 01m 10s)

30 July 2020

  • curprev 00:5200:52, 30 July 2020imported>Stashbot 709,735 bytes +18,579 dzahn@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)

29 July 2020

  • curprev 00:5300:53, 29 July 2020imported>Stashbot 691,156 bytes +25,595 dzahn@cumin1001: conftool action : set/pooled=yes; selector: name=wtp1046.eqiad.wmnet

28 July 2020

  • curprev 00:1900:19, 28 July 2020imported>Stashbot 665,561 bytes +13,224 andrew@cumin1001: conftool action : set/pooled=inactive; selector: name=cloudcephmon1003.eqiad.wmnet

25 July 2020

  • curprev 12:4112:41, 25 July 2020imported>Stashbot 652,337 bytes +620 marostegui@cumin1001: dbctl commit (dc=all): 'Pool db1096:3315 into s5 api afte db1082 crashed T258336', diff saved to https://phabricator.wikimedia.org/P12041 and previous config saved to /var/cache/conftool/dbconfig/20200725-124104-marostegui.json
  • curprev 00:4700:47, 25 July 2020imported>Stashbot 651,717 bytes +3,102 mutante: ganeti - removing disk 3 (20G) from webperf1002. the disks are 0-indexed, so the ones actually mounted are 0 (50G) and 1 (300G) (T257931)

24 July 2020

  • curprev 01:1901:19, 24 July 2020imported>Stashbot 648,615 bytes +12,821 ejegg: updated payments-wiki from 31a3de1130 to c365c136d2

23 July 2020

  • curprev 00:2400:24, 23 July 2020imported>Stashbot 635,794 bytes +17,409 legoktm@deploy1001: Synchronized php-1.35.0-wmf.41/includes/: T258664: Revert "Add a new type of database to the installer from extension" (2/2) (duration: 01m 08s)

21 July 2020

  • curprev 23:3723:37, 21 July 2020imported>Stashbot 618,385 bytes +16,371 ebernhardson@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Bump cirrus MLR models to latest (duration: 01m 06s)
  • curprev 00:0200:02, 21 July 2020imported>Stashbot 602,014 bytes +20,177 ryankemper: Began Elasticsearch reindex job on index `dewiki_content` across [`eqiad`, `codfw`, `cloudelastic`], on `rkemper@mwmaint1002` under tmux session `reindex`. Should complete in <24 hours

19 July 2020

18 July 2020

  • curprev 21:4121:41, 18 July 2020imported>Stashbot 581,291 bytes +539 shdubsh: restart logstash on logstash200[456]
  • curprev 00:1300:13, 18 July 2020imported>Stashbot 580,752 bytes +4,067 dpifke: Performing one-time expiration of ArcLamp files older than 40 days (normal retention is 45 days), to solve disk space issue until either Ganeti issue is solved or compressed logfile support is merged.

16 July 2020

  • curprev 22:1622:16, 16 July 2020imported>Stashbot 576,685 bytes +13,808 mutante: testreduce1001 manually git clone 'scandium' branch of integration/visualdiff into /srv/visualdiff (T257906)

15 July 2020

  • curprev 23:5423:54, 15 July 2020imported>Stashbot 562,877 bytes +11,781 eileen: tools revision changed from 7b6018a16e to 711d671600
  • curprev 01:4501:45, 15 July 2020imported>Stashbot 551,096 bytes +15,235 eileen: tools revision changed from a9e7dc1559 to 7b6018a16e

14 July 2020

  • curprev 01:2301:23, 14 July 2020imported>Stashbot 535,861 bytes +9,041 ryankemper: Started long-running Elasticsearch reindex of `eqiad`, `codfw`, and `cloudelastic`. tmux session `reindex` under `ryankemper` on `mwmaint1002`

11 July 2020

  • curprev 19:1619:16, 11 July 2020imported>Stashbot 526,820 bytes +852 qchris: Restarting Gerrit on gerrit1001 to switch to new gerrit.war and zuul plugin

10 July 2020

  • curprev 21:5221:52, 10 July 2020imported>Stashbot 525,968 bytes +7,988 ryankemper: Started long-running reindex of Elasticsearch indices in `eqiad`, `codfw`, and `dewiki` on `mwmaint1002` under tmux session `reindex` for user `ryankemper`
  • curprev 01:4401:44, 10 July 2020imported>Stashbot 517,980 bytes +10,370 mutante: LDAP - adding coka to wmde and nda (T257038)

9 July 2020

  • curprev 00:5800:58, 9 July 2020imported>Stashbot 507,610 bytes +15,943 cdanis: ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 sudo cumin A:cp 'enable-puppet "cdanis deploying I6c1b646e T256395"'

7 July 2020

  • curprev 22:4122:41, 7 July 2020imported>Stashbot 491,667 bytes +21,022 mutante: new Wikimedia Annual Report 2019 now available on annual.wikimedia.org
  • curprev 00:2200:22, 7 July 2020imported>Stashbot 470,645 bytes +11,069 cstone: civicrm revision changed from a48caf0f37 to d73ee2e73f

5 July 2020

  • curprev 21:5021:50, 5 July 2020imported>Stashbot 459,576 bytes +1,640 qchris: Restarting gerrit on gerrit1001 to pick up new war and jars.

4 July 2020

  • curprev 19:2319:23, 4 July 2020imported>Stashbot 457,936 bytes +722 qchris@deploy1001: Finished deploy [gerrit/gerrit@b78914b]: Bump gitiles to v3.2.2-1-g00c5ca0-with-0e3b533 on gerrit1002 (duration: 00m 08s)

3 July 2020

  • curprev 21:4921:49, 3 July 2020imported>Stashbot 457,214 bytes +4,336 reedy@deploy1001: Synchronized php-1.35.0-wmf.39/extensions/Score/: Sync maintenance script (duration: 00m 58s)

2 July 2020

  • curprev 23:2223:22, 2 July 2020imported>Stashbot 452,878 bytes +4,683 jhuneidi@deploy1001: helmfile [EQIAD] Ran 'sync' command on namespace 'blubberoid' for release 'production' .
  • curprev 00:5300:53, 2 July 2020imported>Stashbot 448,195 bytes +11,373 eileen: tools revision changed from 806e2b4412 to e38f7a83d4

1 July 2020

  • curprev 00:1400:14, 1 July 2020imported>Stashbot 436,822 bytes +12,688 krinkle@deploy1001: Synchronized private/PrivateSettings.php: T254795 - Set $wmgXhguiDBuser and $wmgXhguiDBpasswor (duration: 01m 06s)

29 June 2020

  • curprev 23:2823:28, 29 June 2020imported>Stashbot 424,134 bytes +9,692 eileen: civicrm revision changed from 52a32f2d66 to 391d0fdf75, config revision is f1b4bdb7b7

28 June 2020

  • curprev 21:4321:43, 28 June 2020imported>Stashbot 414,442 bytes +271 krinkle@deploy1001: Synchronized wmf-config/CommonSettings.php: no-op I56eb4a802 (duration: 00m 58s)

27 June 2020

26 June 2020

  • curprev 18:4218:42, 26 June 2020imported>Stashbot 411,866 bytes +4,536 robh: all ulsfo onsite work completed as of 30 minutes ago
  • curprev 01:4101:41, 26 June 2020imported>Stashbot 407,330 bytes +15,922 cdanis: ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕘🍺 sudo cumin A:cp 'enable-puppet "cdanis deploying I6cc5f3e6 T256395"'

25 June 2020

  • curprev 00:4800:48, 25 June 2020imported>Stashbot 391,408 bytes +12,937 twentyafterfour: restart php-fpm on phab1001 to fix T256343

24 June 2020

  • curprev 00:3500:35, 24 June 2020imported>Stashbot 378,471 bytes +8,585 ejegg: restarted fundraising jobs on main CiviCRM box

22 June 2020

  • curprev 23:4123:41, 22 June 2020imported>Stashbot 369,886 bytes +10,612 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: touch for T247330 (duration: 00m 56s)

20 June 2020

  • curprev 22:5622:56, 20 June 2020imported>Stashbot 359,274 bytes +414 cdanis@cumin2001: dbctl commit (dc=all): 'db1088 seems to have crashed', diff saved to https://phabricator.wikimedia.org/P11611 and previous config saved to /var/cache/conftool/dbconfig/20200620-225624-cdanis.json

19 June 2020

  • curprev 18:1018:10, 19 June 2020imported>Stashbot 358,860 bytes +6,598 otto@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Bump eventlogging_Test schema version to 1.1.0 to pick up client_dt - T238230 (duration: 00m 59s)

18 June 2020

  • curprev 22:3022:30, 18 June 2020imported>Stashbot 352,262 bytes +9,559 otto@deploy1001: Synchronized wmf-config/InitialiseSettings.php: EventLogging to EventGate: - SearchSatisfaction on all wikis - T249261 (duration: 00m 56s)

17 June 2020

  • curprev 23:2523:25, 17 June 2020imported>Stashbot 342,703 bytes +17,347 urbanecm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: 0e7079d: Install DiscussionTools on all wikis (attempt 2) (T252264; T253943) (duration: 00m 56s)
  • curprev 01:4301:43, 17 June 2020imported>Stashbot 325,356 bytes +20,465 shdubsh: restart elasticsearch on logstash1011

16 June 2020

  • curprev 00:2800:28, 16 June 2020imported>Stashbot 304,891 bytes +12,502 tstarling@deploy1001: Synchronized wmf-config/CommonSettings.php: limit HTTP client timeout T245170 (duration: 00m 56s)

14 June 2020

  • curprev 13:5113:51, 14 June 2020imported>Stashbot 292,389 bytes +113 qchris: Disabling puppet on gerrit1002 (test instance) to do some more upgrade testing

13 June 2020

  • curprev 21:1221:12, 13 June 2020imported>Stashbot 292,276 bytes +335 qchris: Enabling puppet on gerrit1002 (test instance). Done with testing for today.

12 June 2020

  • curprev 17:4417:44, 12 June 2020imported>Stashbot 291,941 bytes +6,714 herron: restarting logstash1011 elasticsearch instance
  • curprev 01:0201:02, 12 June 2020imported>Stashbot 285,227 bytes +14,139 ejegg: updated payments-wiki from aceddff8b5 to 5fd4eb1519

10 June 2020

  • curprev 23:5523:55, 10 June 2020imported>Stashbot 271,088 bytes +14,140 catrope@deploy1001: Synchronized php-1.35.0-wmf.36/includes/skins/SkinTemplate.php: T255073 (duration: 01m 07s)

9 June 2020

  • curprev 23:1823:18, 9 June 2020imported>Stashbot 256,948 bytes +12,465 Reedy: run namespaceDupes.php --fix for hiwikibooks T254012
  • curprev 00:3900:39, 9 June 2020imported>Stashbot 244,483 bytes +10,746 ejegg: updated payments-wiki from c1d14a5db7 to aceddff8b5

5 June 2020

  • curprev 16:4516:45, 5 June 2020imported>Stashbot 233,737 bytes +4,799 elukey@deploy1001: Finished deploy [analytics/turnilo/deploy@f7e4f78]: Upgrade to 1.24.0 (duration: 00m 11s)

4 June 2020

  • curprev 23:4523:45, 4 June 2020imported>Stashbot 228,938 bytes +11,913 catrope@deploy1001: Synchronized wmf-config/mc.php: Set coalesceKeys=non-global for WANCache on enwiki (duration: 00m 59s)
  • curprev 00:3600:36, 4 June 2020imported>Stashbot 217,025 bytes +14,843 reedy@deploy1001: Synchronized php-1.35.0-wmf.35/includes/specials/SpecialUserrights.php: T254417 T251534 (duration: 01m 06s)

3 June 2020

  • curprev 01:0001:00, 3 June 2020imported>Stashbot 202,182 bytes +10,631 reedy@deploy1001: Synchronized wmf-config/CommonSettings-labs.php: wgLocalVirtualHosts (duration: 01m 06s)

1 June 2020

  • curprev 20:1420:14, 1 June 2020imported>Stashbot 191,551 bytes +5,297 shdubsh: downgrade mtail to rc5 in ulsfo -- T254192

31 May 2020

  • curprev 09:5609:56, 31 May 2020imported>Stashbot 186,254 bytes +199 Urbanecm: mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'Vox Golf' 'Colonel Chicken' (T254068)

29 May 2020

  • curprev 22:3222:32, 29 May 2020imported>Stashbot 186,055 bytes +4,498 bstorm_: updated views on labsdb1010 T252219

28 May 2020

  • curprev 23:4823:48, 28 May 2020imported>Stashbot 181,557 bytes +7,100 jforrester@deploy1001: Synchronized php-1.35.0-wmf.34/skins/Vector/resources/skins.vector.styles/Menu.less: T253912 Hotfix: Cannot rename emptyPortlet to empty-portlet yet (duration: 00m 59s)

27 May 2020

  • curprev 23:2023:20, 27 May 2020imported>Stashbot 174,457 bytes +16,759 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Add autoreviewrestore right to rollbacker group on hiwiki (T252986) (duration: 01m 05s)

26 May 2020

  • curprev 21:3421:34, 26 May 2020imported>Stashbot 157,698 bytes +8,943 krinkle@deploy1001: Synchronized wmf-config/mc.php: I0fb124b3593 (duration: 01m 05s)

25 May 2020

  • curprev 23:3423:34, 25 May 2020imported>Stashbot 148,755 bytes +7,068 ejegg: re-enabled fundraising queue consumers and job runners, except audits, dedupe, and recurring

24 May 2020

  • curprev 17:3617:36, 24 May 2020imported>Stashbot 141,687 bytes +170 gehel: restarting elasticsearch psi on elastic1052

23 May 2020

  • curprev 19:0419:04, 23 May 2020imported>Stashbot 141,517 bytes +906 krinkle@deploy1001: Synchronized php-1.35.0-wmf.31/includes/filerepo/file/LocalFile.php: I0f7e885997d60 (duration: 01m 06s)

22 May 2020

  • curprev 22:4222:42, 22 May 2020imported>Stashbot 140,611 bytes +8,134 krinkle@deploy1001: Synchronized php-1.35.0-wmf.31/includes/filerepo/: Ie19613ef7643a (duration: 01m 06s)

21 May 2020

  • curprev 23:5823:58, 21 May 2020imported>Stashbot 132,477 bytes +8,419 ejegg: updated civicrm from b658fd8233 to 6b1d5902dd

20 May 2020

  • curprev 20:1620:16, 20 May 2020imported>Stashbot 124,058 bytes +10,406 herron: logstash1011:~# kafka-preferred-replica-election --zookeeper conf1004.eqiad.wmnet,conf1005.eqiad.wmnet,conf1006.eqiad.wmnet/kafka/logging-eqiad
  • curprev 00:0500:05, 20 May 2020imported>Stashbot 113,652 bytes +7,654 RoanKattouw: Ran namespaceDupes.php on tiwiki and tiwiktionary for T251287

18 May 2020

  • curprev 23:5023:50, 18 May 2020imported>Stashbot 105,998 bytes +9,614 pt1979@cumin2001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0)

16 May 2020

  • curprev 22:0422:04, 16 May 2020imported>Stashbot 96,384 bytes +1,903 Krinkle: krinkle@mc1022 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet
  • curprev 00:5500:55, 16 May 2020imported>Stashbot 94,481 bytes +6,235 krinkle@deploy1001: Synchronized wmf-config/logging.php: I046868190b472 (duration: 01m 13s)

14 May 2020

  • curprev 23:2423:24, 14 May 2020imported>Stashbot 88,246 bytes +6,005 catrope@deploy1001: Synchronized static/images/project-logos/: Revert temporary 20k logo for vecwiki (T252770) (duration: 01m 06s)
  • curprev 00:5400:54, 14 May 2020imported>Stashbot 82,241 bytes +9,400 foks: change password for "Python eggs"

13 May 2020

  • curprev 00:4300:43, 13 May 2020imported>Stashbot 72,841 bytes +10,801 ryankemper@cumin1001: START - Cookbook sre.wdqs.data-transfer

12 May 2020

  • curprev 00:3700:37, 12 May 2020imported>Stashbot 62,040 bytes +11,598 pt1979@cumin2001: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99)

10 May 2020

  • curprev 12:1812:18, 10 May 2020imported>Stashbot 50,442 bytes +926 marostegui: Start event scheduler on db1115 after a massive delete - T252324

8 May 2020

  • curprev 21:4521:45, 8 May 2020imported>Stashbot 49,516 bytes +4,853 bstorm_: cleaned up wb_terms_no_longer_updated view for testwikidatawiki and testcommonswiki on labsdb1010 T251598
  • curprev 00:3000:30, 8 May 2020imported>Stashbot 44,663 bytes +11,109 brennen@deploy1001: rebuilt and synchronized wikiversions files: Revert all wikis except test to 1.35.0-wmf.30 for T252179

7 May 2020

  • curprev 00:1200:12, 7 May 2020imported>Stashbot 33,554 bytes +5,907 ryankemper@cumin1001: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0)

5 May 2020

  • curprev 23:4423:44, 5 May 2020imported>Stashbot 27,647 bytes +10,916 catrope@deploy1001: Synchronized wmf-config/flaggedrevs.php: Restore the reviewer group on fawiki (T249643) (duration: 01m 06s)

4 May 2020

  • curprev 23:3823:38, 4 May 2020imported>Stashbot 16,731 bytes +12,007 mstyles@deploy1001: Finished deploy [wdqs/wdqs@6518a8d]: v.0.3.26 (duration: 14m 39s)

3 May 2020

  • curprev 22:5222:52, 3 May 2020imported>Stashbot 4,724 bytes +491 Krinkle: scap pull mwmaint1002 and mw2001 for noc.wm.o. – https://gerrit.wikimedia.org/r/593929

2 May 2020

  • curprev 07:4907:49, 2 May 2020imported>Stashbot 4,233 bytes +460 oblivian@cumin1001: conftool action : set/pooled=yes; selector: name=mw13(49|5[0-9]|6[0-2])\.eqiad\.wmnet

1 May 2020

  • curprev 19:5619:56, 1 May 2020imported>Stashbot 3,773 bytes −815,836 rzl@cumin1001: conftool action : set/pooled=no; selector: name=mw13(5[6-9]|6[0-2]).eqiad.wmnet
  • curprev 01:3601:36, 1 May 2020imported>Stashbot 819,609 bytes +7,834 bmansurov@deploy1001: Finished deploy [recommendation-api/deploy@5f47cd7]: Update the recommendation API service (duration: 04m 33s)

30 April 2020

29 April 2020

  • curprev 00:5600:56, 29 April 2020imported>Stashbot 802,927 bytes +15,206 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Enable Growth Study QuickSurvey on enwiki (with sample size 0, for testing) (T248421) (duration: 01m 10s)

27 April 2020

  • curprev 23:2523:25, 27 April 2020imported>Stashbot 787,721 bytes +15,220 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Update logos for tiwiki and tiwiktionary (T150618, T249451) (duration: 00m 57s)

26 April 2020

  • curprev 18:0818:08, 26 April 2020imported>Stashbot 772,501 bytes +140 elukey: powercycle puppetmaster1001 - mgmt serial console not usable, no ssh, racadm getsel doesn't show anything

25 April 2020

  • curprev 10:2310:23, 25 April 2020imported>Stashbot 772,361 bytes +271 addshore: going to restart and probably depool for a short time wdqs1005 as it is in a deadlock T242453

24 April 2020

  • curprev 21:2521:25, 24 April 2020imported>Stashbot 772,090 bytes +562 cdanis@cumin1001: conftool action : set/pooled=true; selector: dnsdisc=wdqs,name=eqiad

23 April 2020

  • curprev 22:0622:06, 23 April 2020imported>Stashbot 771,528 bytes +300 Urbanecm: Perform timeouting rename at enwiki Wikipedia talk:Introduction --> Wikipedia talk:Introduction (historical) using moveBatch.php (request)

22 April 2020

  • curprev 08:5508:55, 22 April 2020imported>Stashbot 771,228 bytes +1,318 Urbanecm: Move User:Wikipedia:Introduction (historical) --> Wikipedia:Introduction (historical) at enwiki using moveBatch.php, on-wiki interface was time-outing
  • curprev 01:4601:46, 22 April 2020imported>Stashbot 769,910 bytes +5,717 milimetric@deploy1001: Started deploy [analytics/refinery@30facc4]: Analytics: another follow-up on the train, jar version bump

20 April 2020

  • curprev 23:2923:29, 20 April 2020imported>Stashbot 764,193 bytes +9,549 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Update project wordmarks and icons (T249047) (duration: 01m 01s)

19 April 2020

  • curprev 16:1916:19, 19 April 2020imported>Stashbot 754,644 bytes +302 reedy@deploy1001: Synchronized wmf-config/LabsServices.php: labs: Move RB traffic to new stretch host (duration: 01m 11s)

18 April 2020

17 April 2020

  • curprev 19:3319:33, 17 April 2020imported>Stashbot 753,607 bytes +5,333 Krinkle: Depool mw1407.eqiad.wmnet for opcache testing. Do not repool without first reverting https://gerrit.wikimedia.org/r/589674.

16 April 2020

  • curprev 22:3422:34, 16 April 2020imported>Stashbot 748,274 bytes +7,894 maryum: reindexing wikis that failed from previous reindex on mwmain1002

15 April 2020

  • curprev 22:1122:11, 15 April 2020imported>Stashbot 740,380 bytes +9,855 mholloway-shell@deploy1001: Synchronized php-1.35.0-wmf.28/extensions/MachineVision: Fix: Initialize categories array for initial images (T250321) (duration: 01m 07s)

14 April 2020

  • curprev 23:2423:24, 14 April 2020imported>Stashbot 730,525 bytes +9,174 AndyRussG: re-enabled thank-you, onimailing and new recurring charge jobs

13 April 2020

  • curprev 23:2423:24, 13 April 2020imported>Stashbot 721,351 bytes +6,995 mdholloway: re-ran extensions/MachineVision/maintenance/withholdImages.php on commonswiki

12 April 2020

  • curprev 11:1111:11, 12 April 2020imported>Stashbot 714,356 bytes +810 vgutierrez: restart ats-tls on cp5008.eqsin.wmnet - T249335

11 April 2020

  • curprev 19:5219:52, 11 April 2020imported>Stashbot 713,546 bytes +836 cdanis@cumin1001: dbctl commit (dc=all): 'slight deweight to db1111', diff saved to https://phabricator.wikimedia.org/P10960 and previous config saved to /var/cache/conftool/dbconfig/20200411-195235-cdanis.json

10 April 2020

  • curprev 21:1221:12, 10 April 2020imported>Stashbot 712,710 bytes +3,913 cdanis@cumin1001: dbctl commit (dc=all): 'db1111 seems overloaded', diff saved to https://phabricator.wikimedia.org/P10954 and previous config saved to /var/cache/conftool/dbconfig/20200410-211202-cdanis.json

9 April 2020

  • curprev 23:4423:44, 9 April 2020imported>Stashbot 708,797 bytes +5,617 jforrester@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Touch and secondary sync of IS for cache-busting (duration: 00m 58s)

8 April 2020

  • curprev 21:2021:20, 8 April 2020imported>Stashbot 703,180 bytes +8,049 jforrester@deploy1001: Synchronized php-1.35.0-wmf.27/extensions/TemplateData/includes/TemplateDataHooks.php: Restore call to OutputPage::setupOOUI() (duration: 01m 07s)

7 April 2020

  • curprev 20:3920:39, 7 April 2020imported>Stashbot 695,131 bytes +13,653 andrewbogott: correction: briefly downtiming ldap-eqiad-replica0 and ldap-eqiad-replica1. I'm trying to investigate a possible split-brain so going to turn ldap off on one, and then the other, to see if behavior changes
  • curprev 00:0100:01, 7 April 2020imported>Stashbot 681,478 bytes +13,583 addshore@deploy1001: Synchronized php-1.35.0-wmf.26/extensions/Wikibase/repo/includes/Store/Sql/DatabaseSchemaUpdater.php: Do not try to drop things when theres no wb_terms table T208425 T249565 cache bust (duration: 01m 01s)

3 April 2020

  • curprev 21:1721:17, 3 April 2020imported>Stashbot 667,895 bytes +4,748 andrewbogott: ugpraded wikitech-static to 1.34.1
  • curprev 00:4200:42, 3 April 2020imported>Stashbot 663,147 bytes +9,745 catrope@deploy1001: Synchronized php-1.35.0-wmf.26/extensions/FlaggedRevs/: Fix logic for determining if pending edits were null (T249277) (duration: 01m 00s)

1 April 2020

  • curprev 22:4422:44, 1 April 2020imported>Stashbot 653,402 bytes +9,945 volker-e@deploy1001: Finished deploy [design/style-guide@4bfe647]: Deploy design/style-guide: (duration: 00m 08s)
  • curprev 00:3900:39, 1 April 2020imported>Stashbot 643,457 bytes +11,449 reedy@deploy1001: Synchronized docroot/mediawiki.org/xml/: Update http and prot rel links to https, fix link to sitelist in MW Core (duration: 01m 06s)

31 March 2020

  • curprev 00:2600:26, 31 March 2020imported>Stashbot 632,008 bytes +8,786 eileen: civicrm revision changed from cf2e2c11c3 to 524b162174, config revision is 708198a154

29 March 2020

  • curprev 08:2408:24, 29 March 2020imported>Stashbot 623,222 bytes +162 elukey: powercycle elastic1059 - mgmt/serial console stuck, no ssh - racadm getsel shows a lot of OEM errors occurred, nothing specific

28 March 2020

27 March 2020

  • curprev 20:5120:51, 27 March 2020imported>Stashbot 622,744 bytes +6,524 ejegg: updated payments-wiki from db618f429d to 1640f5e21e

26 March 2020

  • curprev 23:0723:07, 26 March 2020imported>Stashbot 616,220 bytes +12,328 urbanecm@deploy1001: Synchronized wmf-config/InitialiseSettings.php: SWAT: ce63a4e: Enable wmgUseFooterContactLink for cswiki (T248584; take II) (duration: 00m 57s)
  • curprev 00:0600:06, 26 March 2020imported>Stashbot 603,892 bytes +15,568 catrope@deploy1001: Synchronized wmf-config/InitialiseSettings.php: Enable Special:Investigate on testwiki (T247645) (duration: 03m 14s)

24 March 2020

23 March 2020

  • curprev 21:5021:50, 23 March 2020imported>Stashbot 582,770 bytes +7,770 krinkle@deploy1001: Synchronized docroot/noc/css/vector.css: I627a0ddba5 (duration: 01m 02s)

22 March 2020

  • curprev 23:1923:19, 22 March 2020imported>Stashbot 575,000 bytes +225 reedy@deploy1001: Synchronized wmf-config/InitialiseSettings-labs.php: T248274 (duration: 01m 19s)

20 March 2020

  • curprev 23:1623:16, 20 March 2020imported>Stashbot 574,775 bytes +5,506 dzahn@cumin1001: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0)

19 March 2020

  • curprev 22:1522:15, 19 March 2020imported>Stashbot 569,269 bytes +4,690 mholloway-shell@deploy1001: Finished deploy [mobileapps/deploy@794f099]: Update mobileapps to 99869f45 (duration: 05m 13s)
  • curprev 00:3900:39, 19 March 2020imported>Stashbot 564,579 bytes +8,440 twentyafterfour@deploy1001: rebuilt and synchronized wikiversions files: group0 wikiws to 1.35.0-wmf.24 refs T233872

18 March 2020

  • curprev 00:3100:31, 18 March 2020imported>Stashbot 556,139 bytes +12,787 Amir1: foreachwikiindblist medium deleteEqualMessages.php --delete (T247562)

17 March 2020

  • curprev 00:5700:57, 17 March 2020imported>Stashbot 543,352 bytes +9,300 krinkle@deploy1001: Synchronized php-1.35.0-wmf.23/extensions/Wikibase/lib/includes/Formatters/: Ic77b2c6b33a, T247458 (duration: 01m 12s)

15 March 2020

14 March 2020

  • curprev 08:3308:33, 14 March 2020imported>Stashbot 533,856 bytes +358 elukey: run kafka preferred-replica-election on kafka-jumbo1001 - T247561

13 March 2020

  • curprev 23:1223:12, 13 March 2020imported>Stashbot 533,498 bytes +3,031 bstorm_: rebooting labstore1006 for upgrade to stretch T224583
  • curprev 00:2000:20, 13 March 2020imported>Stashbot 530,467 bytes +13,395 shdubsh: reload prometheus@ops on prometheus1003

12 March 2020

  • curprev 00:5600:56, 12 March 2020imported>Stashbot 517,072 bytes +10,878 ebernhardson@deploy1001: Synchronized php-1.35.0-wmf.22/extensions/CirrusSearch/includes/Maintenance/Reindexer.php: wait around for counts to match up in reindexer before giving up (duration: 01m 08s)

11 March 2020

  • curprev 00:2400:24, 11 March 2020imported>Stashbot 506,194 bytes +18,985 eileen: civicrm revision changed from 35651da117 to c4b81b19b0, config revision is 71c8cda115

9 March 2020

  • curprev 23:2123:21, 9 March 2020imported>Stashbot 487,209 bytes +7,801 catrope@deploy1001: Synchronized wmf-config/throttle.php: Remove expired throttle exemptions (duration: 01m 00s)

8 March 2020

  • curprev 17:5817:58, 8 March 2020imported>Stashbot 479,408 bytes +406 elukey: restart hadoop-yarn-nodemanger on an-worker1087

7 March 2020

  • curprev 12:4812:48, 7 March 2020imported>Stashbot 479,002 bytes +109 reedy@deploy1001: Synchronized wmf-config/throttle.php: T247149 (duration: 01m 07s)
  • curprev 01:3501:35, 7 March 2020imported>Stashbot 478,893 bytes +10,335 reedy@deploy1001: Synchronized wmf-config/throttle.php: tidy up (duration: 00m 56s)

6 March 2020

  • curprev 00:5800:58, 6 March 2020imported>Stashbot 468,558 bytes +21,223 dzahn@cumin1001: START - Cookbook sre.hosts.downtime

5 March 2020

  • curprev 00:5700:57, 5 March 2020imported>Stashbot 447,335 bytes +13,946 twentyafterfour: deploying phabricator-extensions tag release/2020-03-04/1 ( https://phabricator.wikimedia.org/source/phab-extensions/history/wmf%252Fstable/;release/2020-03-04/1 )

4 March 2020

  • curprev 00:2500:25, 4 March 2020imported>Stashbot 433,389 bytes +23,744 ebernhardson@deploy1001: Synchronized php-1.35.0-wmf.21/extensions/WikimediaEvents/modules/ext.wikimediaEvents/searchSatisfaction.js: [cirrus] Match fallback config key with the one used in cirrus (duration: 01m 03s)

3 March 2020

  • curprev 00:5300:53, 3 March 2020imported>Stashbot 409,645 bytes +17,902 jforrester@deploy1001: Synchronized wmf-config/CommonSettings.php: T240055 Update special scandium configuration to load from /srv/parsoid-testing (duration: 00m 58s)

1 March 2020

  • curprev 17:5417:54, 1 March 2020imported>Stashbot 391,743 bytes +926 marostegui: Start replication on db1111 new host on s8 - T246447

29 February 2020

  • curprev 12:3712:37, 29 February 2020imported>Stashbot 390,817 bytes +350 reedy@deploy1001: Synchronized wmf-config/config/viwiki.yaml: T246511 (duration: 00m 56s)

28 February 2020

  • curprev 21:3121:31, 28 February 2020imported>Stashbot 390,467 bytes +6,686 mutante: using planet1001 to manually hack APT sources to test new apt1001.wikimedia.org

27 February 2020

26 February 2020

  • curprev 00:4800:48, 26 February 2020imported>Stashbot 354,983 bytes +15,242 eileen: civicrm revision changed from bec2d6ad9f to 62e62e107c, config revision is c0ef31e2fd

25 February 2020

  • curprev 00:5900:59, 25 February 2020imported>Stashbot 339,741 bytes +14,768 jforrester@deploy1001: Synchronized wmf-config/CommonSettings.php: T238803: Drop ability to load SkinPerPage, EUCopyrightCampaign, and EUCopyrightCampaignSkin (duration: 00m 56s)

23 February 2020

22 February 2020

  • curprev 03:4103:41, 22 February 2020imported>Stashbot 324,738 bytes +596 dzahn@cumin1001: END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0)
  • curprev 00:4500:45, 22 February 2020imported>Stashbot 324,142 bytes +6,187 pt1979@cumin2001: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99)

20 February 2020

19 February 2020

  • curprev 00:4900:49, 19 February 2020imported>Stashbot 291,837 bytes +16,528 pt1979@cumin2001: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99)

17 February 2020

15 February 2020

  • curprev 01:0101:01, 15 February 2020imported>Stashbot 273,144 bytes +176 cdanis: ✔️ cdanis@an-coord1001.eqiad.wmnet ~ 🕗🍺 sudo systemctl restart hive-server2.service ; sudo systemctl restart hive-metastore.service

14 February 2020

  • curprev 23:4223:42, 14 February 2020imported>Stashbot 272,968 bytes +4,182 XenoRyet: updated civicrm from cf86495d44 to 8c77e9e915
  • curprev 00:5300:53, 14 February 2020imported>Stashbot 268,786 bytes +18,037 jforrester@deploy1001: Synchronized wmf-config/InitialiseSettings.php: T245202 cirrus: Move all move_like traffic to codfw (duration: 01m 02s)

13 February 2020

  • curprev 00:5800:58, 13 February 2020imported>Stashbot 250,749 bytes +16,260 pt1979@cumin2001: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99)

12 February 2020

  • curprev 00:4400:44, 12 February 2020imported>Stashbot 234,489 bytes +182,486 jforrester@deploy1001: Synchronized php-1.35.0-wmf.19/includes/page/ImageHistoryPseudoPager.php: T244937 ImageHistoryPseudoPager: Update doQuery() for IndexPager changes (duration: 01m 03s)

11 February 2020

  • curprev 00:2600:26, 11 February 2020imported>Stashbot 52,003 bytes +14,854 ebernhardson@deploy1001: Synchronized php-1.35.0-wmf.18/skins/MinervaNeue: SWAT: Revert: Reduce userContributions icon code (duration: 01m 06s)
(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)