You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Server Admin Log"

Jump to navigation Jump to search

Diff selection: Mark the radio boxes of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | older 500) (20 | 50 | 100 | 250 | 500)
  • curprev 22:44, 4 November 2016imported>Stashbot 663,301 bytes +3,204 godog: stop puppet on einsteinium and tegment to avoid log spam - T150061
  • curprev 01:18, 4 November 2016imported>Stashbot 660,097 bytes +7,635 catrope@terbium: scap failed: IOError [Errno 13] Permission denied: u'/srv/mediawiki-staging/wmf-config/ExtensionMessages-1.29.0-wmf.1.php' (duration: 00m 20s)
  • curprev 00:28, 3 November 2016imported>Stashbot 652,462 bytes +6,304 dereckson@tin: Synchronized php-1.29.0-wmf.1/resources/src/mediawiki.widgets/mw.widgets.TitleWidget.js: Follow-up Id0021594: Remove extra code for redlink suggestions (T149130) (duration: 00m 46s)
  • curprev 01:06, 2 November 2016imported>Stashbot 646,158 bytes +3,818 ejegg: disabled ingenico audit parser
  • curprev 23:46, 31 October 2016imported>Stashbot 642,340 bytes +5,878 thcipriani@tin: Synchronized wmf-config/InitialiseSettings.php: SWAT: MF Beta: Do not move first paragraph before infobox (T145216) (T149389) (duration: 00m 46s)
  • curprev 22:18, 30 October 2016imported>Platonides 636,462 bytes +355 Undo revision 939964 by Stashbot (talk)
  • curprev 22:26, 29 October 2016imported>Stashbot 636,107 bytes +520 reedy@tin: Synchronized php-1.28.0-wmf.23/includes/EditPage.php: Fix for T149473 (duration: 00m 49s)
  • curprev 23:19, 28 October 2016imported>Stashbot 635,587 bytes +2,443 mutante: re-enabled puppet on phab2001 temp, ran puppet. removed 10.64.31.186/21 from eth0, stopped puppet again
  • curprev 00:20, 28 October 2016imported>Stashbot 633,144 bytes +6,333 bd808: Testing logging to SAL via stashbot
  • curprev 01:21, 27 October 2016imported>Labslogbot 626,811 bytes +7,665 mw1208 - service hhvm restart (mutante)
  • curprev 01:13, 26 October 2016imported>Labslogbot 619,146 bytes +7,571 palladium - shutdown -h now (mutante)
  • curprev 23:26, 24 October 2016imported>Zppix 611,575 bytes +5,588 Rm spam
  • curprev 15:37, 22 October 2016imported>Labslogbot 605,987 bytes +506 oblivian@puppetmaster1001 conftool action : set/pooled=no; selector: name=cp1052.eqiad.wmnet (logmsgbot)
  • curprev 23:45, 21 October 2016imported>Labslogbot 605,481 bytes +4,290 depooling maps1002 (by running "depool" on the server itself) (mutante)
  • curprev 23:51, 20 October 2016imported>Labslogbot 601,191 bytes +8,920 rebooting eeden (ns2) for kernel (bblack)
  • curprev 01:17, 20 October 2016imported>Labslogbot 592,271 bytes +5,838 legoktm@mira Synchronized wmf-config/InitialiseSettings.php: Revert Enable AbuseFilterCachingParser by default - T148673 (duration: 00m 51s) (logmsgbot)
  • curprev 01:24, 19 October 2016imported>Matiia 586,433 bytes +3,421 Undo revision 911491 by Labslogbot (talk)
  • curprev 00:48, 18 October 2016imported>Labslogbot 583,012 bytes +3,523 restarting API hhvms with >40% mem usage via salt every 10 minutes in a loop from here forward. screen session on neodymium, named api-hhvm-restarts (bblack)
  • curprev 20:59, 16 October 2016imported>Labslogbot 579,489 bytes +441 ladsgroup@scb[12]00[12]: sudo service celery-ores-worker restart (Amir1)
  • curprev 06:32, 15 October 2016imported>Labslogbot 579,048 bytes +349 tstarling@mira Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 02m 30s) (logmsgbot)
  • curprev 21:49, 14 October 2016imported>Labslogbot 578,699 bytes +3,417 deployed & restarted kartotherian, all's good now (yurik)
  • curprev 00:17, 14 October 2016imported>Labslogbot 575,282 bytes +6,278 reedy@mira Synchronized php-1.28.0-wmf.22/extensions/PageAssessments: [extensions/PageAssessments] Only update assessment data when talk pages are saved (duration: 00m 51s) (logmsgbot)
  • curprev 23:41, 12 October 2016imported>Labslogbot 569,004 bytes +5,874 updated civicrm settings to revision a7c7919cf6ff7a4722e0f98aca90c43edfd3ac9c (ejegg)
  • curprev 00:02, 12 October 2016imported>Labslogbot 563,130 bytes +8,809 ebernhardson@mira Synchronized php-1.28.0-wmf.22/extensions/CirrusSearch/: SWAT CirrusSearch Add completion support to ClusterOverride, Remove position_increment_gap on source_text.trigram (duration: 00m 58s) (logmsgbot)
  • curprev 21:29, 10 October 2016imported>Labslogbot 554,321 bytes +5,295 reedy@tin Synchronized wmf-config/InitialiseSettings.php: Add upload_by_url right to Commons bots (duration: 00m 50s) (logmsgbot)
  • curprev 09:07, 9 October 2016imported>Labslogbot 549,026 bytes +516 chmod o+r /var/lib/varnish/frontend/_.vsm and /var/lib/varnish/cp2008/_.vsm on cp2008 to avoid gmond errors (elukey)
  • curprev 19:48, 8 October 2016imported>Labslogbot 548,510 bytes +529 cp2008 Strongswan failures for both ipv4 and ipv6 across a larg number (all?) hosts (bd808)
  • curprev 23:16, 7 October 2016imported>Labslogbot 547,981 bytes +3,727 test morebots (Krenair)
  • curprev 00:54, 7 October 2016imported>Labslogbot 544,254 bytes +9,114 https://olo.wikipedia.org has been successfully created (T146612). (Dereckson)
  • curprev 00:15, 6 October 2016imported>Labslogbot 535,140 bytes +10,057 phabricator update complete and service is restored (twentyafterfour)
  • curprev 00:05, 5 October 2016imported>Labslogbot 525,083 bytes +6,817 maxsem@tin Synchronized wmf-config/mobile.php: https://gerrit.wikimedia.org/r/#/c/314206/1 (duration: 00m 49s) (logmsgbot)
  • curprev 01:09, 4 October 2016imported>Labslogbot 518,266 bytes +9,784 echo 'https://noc.wikimedia.org/' | mwscript purgeList.php (Dereckson)
  • curprev 07:29, 2 October 2016imported>Labslogbot 508,482 bytes +333 silencing wdqs response time alerts, it is flapping, related to traffic - T147130 (gehel)
  • curprev 11:03, 1 October 2016imported>Labslogbot 508,149 bytes +274 ladsgroup@terbium:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki Gautehuus Neuraxıs (Amir1)
  • curprev 22:06, 30 September 2016imported>Labslogbot 507,875 bytes +620 Re-run mwscript deleteEqualMessages.php on all wikis it was previously run on (T45917) (Krinkle)
  • curprev 00:29, 30 September 2016imported>Labslogbot 507,255 bytes +3,307 Manually updated the DB to fix already-broken cases caused by since-fixed T138310 (matt_flaschen)
  • curprev 01:04, 29 September 2016imported>Labslogbot 503,948 bytes +1,816 rolled smashpig back because JSON_UNESCAPED_UNICODE is unavailable in php5.3 and don't want consumers to explode (cwd)
  • curprev 01:20, 28 September 2016imported>Labslogbot 502,132 bytes +340 updating CiviCRM from 26e5214e66f2e4b5de89c27dd5121dfde43269e2 to 88000acc826592e535491bde6a47f8976deaf07a (eileen)
  • curprev 19:37, 26 September 2016imported>Labslogbot 501,792 bytes +768 enabling awight_test5 banner at 1% of nlwiki (awight)
  • curprev 16:26, 25 September 2016imported>Labslogbot 501,024 bytes +594 deploying 9cc2009 on ORES nodes (T146581) (Amir1)
  • curprev 19:30, 24 September 2016imported>Labslogbot 500,430 bytes +459 hhvm 1283-1290 rolling restart (ema)
  • curprev 22:05, 23 September 2016imported>Labslogbot 499,971 bytes +3,722 Deployed patch for T146425 (matt_flaschen)
  • curprev 00:38, 23 September 2016imported>Labslogbot 496,249 bytes +6,920 mw1224 apache stuck, not restarting for now in case someone wants to investigate later. possibly T89912? (Krenair)
  • curprev 01:28, 22 September 2016imported>Labslogbot 489,329 bytes +7,092 Rebooting iridium to apply kernel update (twentyafterfour)
  • curprev 01:15, 21 September 2016imported>Labslogbot 482,237 bytes +4,787 updated SmashPig to 285d8ce25f22d8538c693c1b3c3e7ce9e688f22c (awight)
  • curprev 00:43, 20 September 2016imported>Labslogbot 477,450 bytes +5,677 wtp2019 - down again, powercycled, probably damaged RAM (mutante)
  • curprev 20:37, 18 September 2016imported>Labslogbot 471,773 bytes +1,940 restart upload backend: cp3036 (bblack)
  • curprev 16:40, 17 September 2016imported>Labslogbot 469,833 bytes +732 rolling restart of HHVM on part fo the API cluster in eqiad, T133674 (_joe_)
  • curprev 23:09, 16 September 2016imported>Labslogbot 469,101 bytes +3,396 titanium - shutdown -h now (mutante)
  • curprev 01:28, 16 September 2016imported>Labslogbot 465,705 bytes +6,685 mw1294 - down and frozen, powercycled (mutante)
  • curprev 23:44, 14 September 2016imported>Labslogbot 459,020 bytes +6,342 dereckson@tin Synchronized php-1.28.0-wmf.19/extensions/Kartographer/tests/phpunit/KartographerTest.php: Always serve all the data on preview (T145615, 2/2, no-op part) (duration: 00m 50s) (logmsgbot)
  • curprev 23:32, 13 September 2016imported>Labslogbot 452,678 bytes +6,494 catrope@tin Synchronized php-1.28.0-wmf.18/resources/lib/moment/locale: T145382 (duration: 00m 47s) (logmsgbot)
  • curprev 01:27, 13 September 2016imported>Labslogbot 446,184 bytes +7,455 aaron@tin Synchronized php-1.28.0-wmf.18/extensions/GlobalUsage: 1843a856c7c648a310a1983d53139e5b79ea6585 (duration: 00m 48s) (logmsgbot)
  • curprev 01:28, 12 September 2016imported>Labslogbot 438,729 bytes +579 upgrade cp3035 to varnish 4 T131502 (ema)
  • curprev 22:30, 10 September 2016imported>Labslogbot 438,150 bytes +546 T144826: Restarting Cassandra on restbase2004-b.codfw.wmnet (scrub complete, re-joining cluster) (urandom)
  • curprev 19:29, 9 September 2016imported>Labslogbot 437,604 bytes +2,578 demon@tin Synchronized wmf-config/wikitech.php: bizarro config loading (duration: 00m 46s) (logmsgbot)
  • curprev 01:07, 9 September 2016imported>Labslogbot 435,026 bytes +6,868 aaron@tin Synchronized tests/Defines.php: (no message) (duration: 00m 46s) (logmsgbot)
  • curprev 00:46, 8 September 2016imported>Labslogbot 428,158 bytes +9,118 krenair@tin Synchronized php-1.28.0-wmf.18/extensions/VisualEditor/extension.json: https://gerrit.wikimedia.org/r/#/c/309213/ (duration: 00m 46s) (logmsgbot)
  • curprev 01:07, 7 September 2016imported>Labslogbot 419,040 bytes +5,247 Updated Wikidata's property suggester with data from Monday's json dump and applied the T132839 workarounds (hoo)
  • curprev 23:27, 5 September 2016imported>Labslogbot 413,793 bytes +5,651 dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Set wgMathFileBackend to false for wikitech wikis (T126628) (duration: 00m 48s) (logmsgbot)
  • curprev 20:11, 4 September 2016imported>Labslogbot 408,142 bytes +497 re-enabled ircecho on neon (elukey)
  • curprev 14:24, 3 September 2016imported>Labslogbot 407,645 bytes +320 depool cp4005 (ema)
  • curprev 22:31, 2 September 2016imported>Labslogbot 407,325 bytes +1,769 aaron@tin Synchronized php-1.28.0-wmf.17/includes/page/WikiPage.php: No-op sync - 880c7f9526610bfd396dddb14f7772f9247850cb (duration: 00m 49s) (logmsgbot)
  • curprev 01:06, 2 September 2016imported>Labslogbot 405,556 bytes +8,920 dereckson@tin Synchronized wmf-config/CommonSettings.php: Customize wgMathDirectory for wikitech (T126628, 2/2) (duration: 00m 46s) (logmsgbot)
  • curprev 01:00, 1 September 2016imported>Labslogbot 396,636 bytes +6,146 reboot db1025 for kernel update (Jeff_Green)
  • curprev 00:06, 31 August 2016imported>Labslogbot 390,490 bytes +6,547 catrope@tin Synchronized php-1.28.0-wmf.17/extensions/Flow: SWAT (duration: 01m 09s) (logmsgbot)
  • curprev 23:47, 29 August 2016imported>Labslogbot 383,943 bytes +7,793 stop nfs server on labstore1001 to try to fix scratch mount (chasemp)
  • curprev 16:51, 28 August 2016imported>Labslogbot 376,150 bytes +312 Ran T132839-Workarounds.sh from my home in terbium (see T132839) (hoo)
  • curprev 02:30, 27 August 2016imported>Labslogbot 375,838 bytes +234 l10nupdate@tin ResourceLoader cache refresh completed at Sat Aug 27 02:30:35 UTC 2016 (duration 5m 50s) (logmsgbot)
  • curprev 22:51, 26 August 2016imported>Labslogbot 375,604 bytes +218 from a few hours ago: (08:42:47 μμ) bd808: !log Updated striker to fix T143956 (apergos)
  • curprev 15:13, 25 August 2016imported>Labslogbot 375,386 bytes +2,010 pooling cp4005 backend (varnish 4 cache_upload) T131502 (ema)
  • curprev 01:08, 25 August 2016imported>Labslogbot 373,376 bytes +7,783 radium apt-get autoremove; apt-get upgrade (openssh, openssl, passwd, sudo, libpam, libc6 :) (mutante)
  • curprev 23:45, 23 August 2016imported>Labslogbot 365,593 bytes +5,161 dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Enable Related Articles on fr.wikinews (T143480) (duration: 00m 53s) (logmsgbot)
  • curprev 23:58, 22 August 2016imported>Labslogbot 360,432 bytes +6,741 dereckson@tin Synchronized php-1.28.0-wmf.15/extensions/AbuseFilter/Views/: Let abusefilter-modify users see history of hidden filters (gerrit:305596+gerrit:306077, T143365) (duration: 00m 50s) (logmsgbot)
  • curprev 15:01, 21 August 2016imported>Labslogbot 353,691 bytes +315 Run deleteEqualMessages maintenance script on urwiki (T45917) (Dereckson)
  • curprev 13:34, 20 August 2016imported>Labslogbot 353,376 bytes +343 rolling, depooled restarts for varnish-frontends: all text, upload in esams, eqiad, codfw. (bblack--)
  • curprev 22:58, 19 August 2016imported>Labslogbot 353,033 bytes +1,846 works fine, no more "bastiononly" group - all users get automatically added on bastions (mutante)
  • curprev 00:31, 19 August 2016imported>Labslogbot 351,187 bytes +7,084 rollback civicrm to a30fc0aa2c2dc3a7a9a0b09bef19d112bdf5f98e (awight)
  • curprev 00:27, 18 August 2016imported>Labslogbot 344,103 bytes +6,174 maxsem@tin Finished scap: https://gerrit.wikimedia.org/r/#/c/305424/ (duration: 52m 22s) (logmsgbot)
  • curprev 23:57, 16 August 2016imported>Labslogbot 337,929 bytes +4,182 krinkle@tin Synchronized php-1.28.0-wmf.15/includes/OutputPage.php: 653232f90605 (duration: 00m 52s) (logmsgbot)
  • curprev 00:21, 16 August 2016imported>Labslogbot 333,747 bytes +4,141 demon@tin Synchronized wmf-config/InitialiseSettings.php: remove utc timezone overrides (duration: 00m 48s) (logmsgbot)
  • curprev 17:12, 14 August 2016imported>Labslogbot 329,606 bytes +387 bumping cr2-knams<->cr1-eqiad OSPF/OSPF3 metric to 1820 (thus activating the new cr2-esams<->cr2-eqiad link which has a metric of 840) (paravoid)
  • curprev 20:59, 13 August 2016imported>Labslogbot 329,219 bytes +1,885 labtestcontrol2001: restarted mysql to unbreak labtesthorizon login again. we really need to figure out why this becomes necessary (Krenair)
  • curprev 20:13, 12 August 2016imported>Labslogbot 327,334 bytes +1,907 running FLUSHALL on oresrdb1001 T142857 (halfak)
  • curprev 00:51, 12 August 2016imported>Labslogbot 325,427 bytes +3,649 deployed kartotherian & tilerator. maps100[134].eqiad are still down (non production) (yurik)
  • curprev 00:41, 11 August 2016imported>Labslogbot 321,778 bytes +5,622 aaron@tin Synchronized php-1.28.0-wmf.14/maintenance/purgeChangedFiles.php: 91baa668219d8a77b1e300c257e969b04b4b1e3d (duration: 00m 47s) (logmsgbot)
  • curprev 00:42, 10 August 2016imported>Labslogbot 316,156 bytes +6,827 reedy@tin Synchronized php-1.28.0-wmf.14/extensions/timeline/Timeline.body.php: (no message) (duration: 00m 48s) (logmsgbot)
  • curprev 01:01, 9 August 2016imported>Labslogbot 309,329 bytes +12,484 krinkle@tin Synchronized wmf-config/InitialiseSettings.php: coding style (duration: 00m 56s) (logmsgbot)
  • curprev 23:39, 7 August 2016imported>Labslogbot 296,845 bytes +2,130 reedy@tin Synchronized wmf-config: last 2 to wfLoadExtension (duration: 00m 59s) (logmsgbot)
  • curprev 23:09, 6 August 2016imported>Labslogbot 294,715 bytes +661 cleaned and re-accepted salt-key for labvirt1014, minion back up now (yuvipanda)
  • curprev 01:02, 6 August 2016imported>Labslogbot 294,054 bytes +6,824 re-imaging labvirt1014 (andrewbogott)
  • curprev 23:37, 4 August 2016imported>Labslogbot 287,230 bytes +7,948 maxsem@tin Synchronized php-1.28.0-wmf.13/extensions/Echo: https://gerrit.wikimedia.org/r/#/c/303095/ https://gerrit.wikimedia.org/r/#/c/303099/ (duration: 00m 54s) (logmsgbot)
  • curprev 00:19, 4 August 2016imported>Labslogbot 279,282 bytes +11,979 revert paymentswiki config to 70b2ff90d9c8f19716f2e9c07a8dc8cfa17991ca (awight)
  • curprev 01:17, 3 August 2016imported>Labslogbot 267,303 bytes +6,894 revert paymentswiki config change, new commit 4d0090f50d515047c5e4dfbe565cd04bdd31801d (awight)
  • curprev 23:31, 1 August 2016imported>Labslogbot 260,409 bytes +4,823 Upgrading CiviCRM from d657255e1edebeccfc0a03bea70b78eb11375cf8 to 4904c4aae3565b65d5f37ecb827ea26c930b72d6 (eileen)
  • curprev 18:07, 31 July 2016imported>Labslogbot 255,586 bytes +486 aaron@tin Synchronized php-1.28.0-wmf.12/includes/page/WikiPage.php: 061737668d729ba1a76 (duration: 00m 24s) (logmsgbot)
  • curprev 23:36, 30 July 2016imported>Labslogbot 255,100 bytes +302 deleted 7 files from server for legal compliance (Jamesofur)
  • curprev 23:56, 29 July 2016imported>Labslogbot 254,798 bytes +2,138 hardreset xenon (YuviPanda)
  • curprev 23:29, 28 July 2016imported>Labslogbot 252,660 bytes +4,038 updated payments-wiki from 2d9dd79507a42ced0a99bde87b3c45b804610e40 to 3a724bfb1a3e20e17b5886dae0ba7572020abd6b4 (ejegg)
  • curprev 23:45, 27 July 2016imported>Labslogbot 248,622 bytes +5,885 dereckson@tin Synchronized wmf-config/InitialiseSettings-labs.php: Test numeric sorting on Beta Cluster (Gerrit:301520, labs only, no-op in prod) (duration: 00m 23s) (logmsgbot)
  • curprev 01:05, 27 July 2016imported>Labslogbot 242,737 bytes +6,018 rolled back paymentswiki from 79d2b67067fd7e579372b63e0d619eccfa3b9143 to 79cb53998c41f72d0fa49130ed1f66dc112b478c (ejegg)
  • curprev 01:11, 26 July 2016imported>Labslogbot 236,719 bytes +3,677 deploying from 2d9817b to a291da1 for ores in scb nodes (Amir1)
  • curprev 01:17, 25 July 2016imported>Labslogbot 233,042 bytes +828 scandium: migrating zuul-merger repos to lead find /srv/ssd/zuul/git -path '*/.git/config' -print -execdir sed -i -e 's/ytterbium.wikimedia.org/lead.wikimedia.org/' config \; (hashar)
  • curprev 15:38, 23 July 2016imported>Labslogbot 232,214 bytes +609 stop swift in esams test cluster, lots of logging from there (godog)
  • curprev 01:02, 23 July 2016imported>Labslogbot 231,605 bytes +2,844 tgr@tin Synchronized php-1.28.0-wmf.11/extensions/CentralAuth/includes/CentralAuthPlugin.php: T141160 (duration: 00m 29s) (logmsgbot)
  • curprev 00:53, 22 July 2016imported>Labslogbot 228,761 bytes +8,317 Restarted elasticsearch on logstash1003; couldn't find master (even though the master thought 1003 was fine) (bd808)
  • curprev 23:43, 20 July 2016imported>Labslogbot 220,444 bytes +6,321 maxsem@tin Synchronized wmf-config/InitialiseSettings-labs.php: SWAT, no-op (duration: 00m 24s) (logmsgbot)
  • curprev 01:29, 20 July 2016imported>Labslogbot 214,123 bytes +5,376 rhodium, new puppetmaster, add to salt (mutante)
  • curprev 23:55, 18 July 2016imported>Labslogbot 208,747 bytes +5,983 maxsem@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/292622/ (duration: 00m 25s) (logmsgbot)
  • curprev 10:31, 17 July 2016imported>Labslogbot 202,764 bytes +284 restart slapd on serpens - T130593 (godog)
  • curprev 23:16, 16 July 2016imported>Labslogbot 202,480 bytes +825 testing that the SAL bot is still working (Krenair)
  • curprev 21:12, 15 July 2016imported>Labslogbot 201,655 bytes +1,933 reenable donation queue (awight)
  • curprev 23:11, 14 July 2016imported>Labslogbot 199,722 bytes +3,743 ebernhardson@tin Synchronized php-1.28.0-wmf.10/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: T137169: Turn of TextCat A/B test (duration: 00m 34s) (logmsgbot)
  • curprev 00:36, 14 July 2016imported>Labslogbot 195,979 bytes +8,669 Ran backfillReadBundles on labtestwiki (matt_flaschen)
  • curprev 00:56, 13 July 2016imported>Labslogbot 187,310 bytes +6,645 Restarted grrrit-wm (ori)
  • curprev 23:49, 11 July 2016imported>Labslogbot 180,665 bytes +6,405 maxsem@tin Synchronized php-1.28.0-wmf.9/resources/: https://gerrit.wikimedia.org/r/#/c/298402/ (duration: 00m 28s) (logmsgbot)
  • curprev 02:25, 10 July 2016imported>Labslogbot 174,260 bytes +233 l10nupdate@tin ResourceLoader cache refresh completed at Sun Jul 10 02:25:55 UTC 2016 (duration 5m 43s) (logmsgbot)
  • curprev 19:54, 9 July 2016imported>Labslogbot 174,027 bytes +568 restarted logstash on logstash1003 for de-dot plugin update (T136001) (bd808)
  • curprev 00:28, 9 July 2016imported>Labslogbot 173,459 bytes +4,623 analytics1027 - icinga said puppet fail, just ran it, recovery, same on neon.. something kafka graphite checks (mutante)
  • curprev 01:26, 8 July 2016imported>Labslogbot 168,836 bytes +10,839 gerrit: readded robots.txt to ytterbium for now (ostriches)
  • curprev 01:24, 7 July 2016imported>Labslogbot 157,997 bytes +8,871 rebooting labvirt1011 (because it is acting crazy) (andrewbogott)
  • curprev 00:23, 6 July 2016imported>Labslogbot 149,126 bytes +8,056 maxsem@tin Synchronized wmf-config: No-op (duration: 00m 37s) (logmsgbot)
  • curprev 20:28, 4 July 2016imported>Labslogbot 141,070 bytes +3,374 removing /tmp/joal/sstables on all analytics10* hosts (jynus)
  • curprev 19:27, 3 July 2016imported>Labslogbot 137,696 bytes +611 Ran namespaceDupes --fix on gomwiki (Reedy)
  • curprev 19:15, 2 July 2016imported>Labslogbot 137,085 bytes +319 Deployed hotfix to phabricator. Restarted apache2 on iridium (twentyafterfour)
  • curprev 22:23, 1 July 2016imported>Labslogbot 136,766 bytes +2,894 krinkle@tin Synchronized php-1.28.0-wmf.8/extensions/WikimediaEvents/extension.json: T128115 (duration: 00m 37s) (logmsgbot)
  • curprev 01:08, 1 July 2016imported>Labslogbot 133,872 bytes +7,296 ori@tin Synchronized wmf-config/InitialiseSettings.php: Ie8a71af5: HD logo for en.wiktionary (2/2) (duration: 00m 27s) (logmsgbot)
  • curprev 01:18, 30 June 2016imported>Labslogbot 126,576 bytes +8,913 iridium back up, on 3.13.0-91 (mutante)
  • curprev 00:02, 29 June 2016imported>Labslogbot 117,663 bytes +5,544 twentyafterfour@tin Finished scap: sync new branch, testwiki to php-1.28.0-wmf.8 refs T137492 (duration: 51m 59s) (logmsgbot)
  • curprev 00:26, 28 June 2016imported>Labslogbot 112,119 bytes +4,764 dereckson@tin Synchronized php-1.28.0-wmf.7/includes/api/ApiMain.php: UsageException to try to catch T138585 issue (duration: 00m 27s) (logmsgbot)
  • curprev 02:52, 26 June 2016imported>Labslogbot 107,355 bytes +325 l10nupdate@tin ResourceLoader cache refresh completed at Sun Jun 26 02:52:48 UTC 2016 (duration 6m 19s) (logmsgbot)
  • curprev 09:37, 25 June 2016imported>Labslogbot 107,030 bytes +404 install2001 killing ganglia aggregator processes, running puppet, for debugging (mutante)
  • curprev 01:07, 25 June 2016imported>Labslogbot 106,626 bytes +2,426 sign labstore1005 puppet certs and bootstrap the server (chasemp)
  • curprev 01:22, 24 June 2016imported>Labslogbot 104,200 bytes +5,709 stream.wikimedia.org (RCStream) DNS moved to cache_misc termination. If anyone reports bugs with rcstream services, revert https://gerrit.wikimedia.org/r/295385 (bblack)
  • curprev 23:25, 22 June 2016imported>Labslogbot 98,491 bytes +4,962 maxsem@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/295560/ (duration: 00m 25s) (logmsgbot)
  • curprev 23:14, 21 June 2016imported>Labslogbot 93,529 bytes +5,706 updated/restarted kartotherian & tilerator - https://gerrit.wikimedia.org/r/#/c/295440/ https://gerrit.wikimedia.org/r/#/c/295441/ (yurik)
  • curprev 23:22, 20 June 2016imported>Labslogbot 87,823 bytes +5,075 `mwscript namespaceDupes.php ptwikinews --fix` (T138230). Some links and revisions are still to fix. (Dereckson)
  • curprev 12:29, 19 June 2016imported>Labslogbot 82,748 bytes +415 restarted hhvm on mw1138 - trace in /tmp/hhvm.25048.bt, hhvm killed by OOM (elukey)
  • curprev 02:32, 18 June 2016imported>Labslogbot 82,333 bytes +233 l10nupdate@tin ResourceLoader cache refresh completed at Sat Jun 18 02:32:26 UTC 2016 (duration 6m 18s) (logmsgbot)
  • curprev 21:21, 17 June 2016imported>Labslogbot 82,100 bytes +2,746 Reenabling puppet and resetting configuration on xenon.eqiad.wmnet : T137419 (urandom)
  • curprev 23:44, 16 June 2016imported>Labslogbot 79,354 bytes +3,948 ebernhardson@tin Synchronized php-1.28.0-wmf.6/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: T137167: TextCat A/B test for Language Identification (duration: 00m 25s) (logmsgbot)
  • curprev 00:57, 16 June 2016imported>Labslogbot 75,406 bytes +7,696 puppet disabled on iridium because https://gerrit.wikimedia.org/r/#/c/294653/ needs to merge (hotfix in preamble.php which puppet will undo if it's allowed to run) (twentyafterfour)
  • curprev 23:40, 14 June 2016imported>Labslogbot 67,710 bytes +8,126 ori@tin Synchronized php-1.28.0-wmf.6/resources/src/mediawiki.action/mediawiki.action.edit.stash.js: Idfad8407c8e: Improve client-side edit stash change detection (duration: 00m 25s) (logmsgbot)
  • curprev 23:50, 13 June 2016imported>Labslogbot 59,584 bytes +6,189 dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Add ORES to whitelisted beta features (T130211) (duration: 00m 23s) (logmsgbot)
  • curprev 02:28, 12 June 2016imported>Labslogbot 53,395 bytes +110 mwdeploy@tin scap sync-l10n completed (1.28.0-wmf.5) (duration: 12m 24s) (logmsgbot)
  • curprev 03:14, 11 June 2016imported>Labslogbot 53,285 bytes +530 ori@tin Synchronized php-1.28.0-wmf.5/includes/parser/CacheTime.php: remove ad-hoc logging of updateCacheExpiry(0) traces (duration: 00m 23s) (logmsgbot)
  • curprev 01:01, 11 June 2016imported>Labslogbot 52,755 bytes +3,675 rutherfordium ganeti lockup, gnt-instance console .. and it recovered (mutante)
  • curprev 00:56, 10 June 2016imported>Labslogbot 49,080 bytes +8,670 ran mwscript maintenance/updateCollation.php --wiki=tawiki --force (kaldari)
  • curprev 22:44, 8 June 2016imported>Labslogbot 40,410 bytes +5,708 tgr@tin Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 for reals T135504 (duration: 00m 25s) (logmsgbot)
  • curprev 00:11, 8 June 2016imported>Labslogbot 34,702 bytes +5,927 update fundraising-tools from b2425aef2154d6b689900f4848cca02880321230 to 28bc2da677caa795c58f906db76a1f8d612ac899 (awight_)
  • curprev 01:10, 7 June 2016imported>Labslogbot 28,775 bytes +6,011 aude@tin Synchronized php-1.28.0-wmf.4/extensions/Wikidata: Fix bug (T136093) in display of labels after edit (duration: 02m 03s) (logmsgbot)
  • curprev 14:55, 5 June 2016imported>Labslogbot 22,764 bytes +315 `mwscript initSiteStats.php --wiki csbwiki --update` (T137060) (Dereckson)
  • curprev 20:18, 4 June 2016imported>Labslogbot 22,449 bytes +626 rebooting mw1135, unresponsive to ssh or console login (apergos)
  • curprev 22:57, 3 June 2016imported>Labslogbot 21,823 bytes −613,635 Purged https://en.wikipedia.org/static/images/project-logos/bawiki.png (Krinkle)
  • curprev 01:09, 3 June 2016imported>Labslogbot 635,458 bytes +7,001 bromine - puppet currently stopped needs some permission fixes for release upload (mutante)
  • curprev 01:25, 2 June 2016imported>Labslogbot 628,457 bytes +11,487 dereckson@tin Synchronized wmf-config/CommonSettings.php: Set $wgSpamBlacklistEventLogging to true on testwiki (duration: 00m 22s) (logmsgbot)
  • curprev 00:04, 1 June 2016imported>Labslogbot 616,970 bytes +6,601 Started `mwscript updateCollation.php itwiki --previous-collation=uppercase` on Terbium (T136647) (Dereckson)
  • curprev 23:04, 30 May 2016imported>Labslogbot 610,369 bytes +3,005 catrope@tin Synchronized wmf-config/InitialiseSettings.php: Update BetaFeatures whitelist (duration: 00m 32s) (logmsgbot)
  • curprev 02:58, 29 May 2016imported>Labslogbot 607,364 bytes +299 Bootsrapping restbase2004-b.codfw.wmet : T134016 (urandom)
  • curprev 19:47, 28 May 2016imported>Labslogbot 607,065 bytes +451 Updated Wikidata's property suggester with data from Monday's json dump and removed the external identifiers as a workaround for T132839 (hoo)
  • curprev 23:58, 27 May 2016imported>Labslogbot 606,614 bytes +2,556 Forcing a Puppet run on logstash* (ori)
  • curprev 01:18, 27 May 2016imported>Labslogbot 604,058 bytes +6,007 updated payments from e6807395d7687d521070b83d159b77b242e5c04f to de0398a244094f2bad6bc70eefce8388a616e575 (ejegg)
  • curprev 00:57, 26 May 2016imported>Labslogbot 598,051 bytes +5,040 dereckson@tin Synchronized php-1.28.0-wmf.3/extensions/WikimediaMessages/i18n/wikimedia: Add i18n messages for new Support and Safety group (duration: 00m 26s) (logmsgbot)
  • curprev 00:22, 25 May 2016imported>Labslogbot 593,011 bytes +5,949 dereckson@tin Synchronized /srv/mediawiki-staging/php-1.28.0-wmf.3/extensions/Flow/maintenance/FlowRemoveOldTopics.php: Don't assume workflows/revisions are inserted in chronological order (T119509) (duration: 00m 23s) (logmsgbot)
  • curprev 23:36, 23 May 2016imported>Labslogbot 587,062 bytes +6,716 dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Enable Flow beta feature on frwikivoyage (T135702) (duration: 00m 25s) (logmsgbot)
  • curprev 17:16, 22 May 2016imported>Labslogbot 580,346 bytes +416 defragmenting db1028 (jynus)
  • curprev 02:32, 21 May 2016imported>Labslogbot 579,930 bytes +215 l10nupdate@tin ResourceLoader cache refresh completed at Sat May 21 02:32:13 UTC 2016 (duration 8m 44s) (logmsgbot)
  • curprev 01:24, 21 May 2016imported>Labslogbot 579,715 bytes +3,599 updated payments from da453121d4086d6c1218d603149e4f568db321b5 to 76b5c559e08b340fb3d66254b27f624a9c0a4b95 (ejegg)
  • curprev 00:52, 20 May 2016imported>Labslogbot 576,116 bytes +8,740 stashbot test (T122690) (mutante)
  • curprev 01:22, 19 May 2016imported>Labslogbot 567,376 bytes +7,063 Phabricator upgrade completed and service restored. (twentyafterfour)
  • curprev 01:25, 18 May 2016imported>Labslogbot 560,313 bytes +6,544 dereckson@tin Started scap: Echo: Bring back Echo email messages (Gerrit:289346, for wmf1) (logmsgbot)
  • curprev 01:13, 17 May 2016imported>Labslogbot 553,769 bytes +6,311 krenair@tin Synchronized docroot/noc: https://gerrit.wikimedia.org/r/#/c/289118/ (duration: 00m 27s) (logmsgbot)
  • curprev 02:28, 14 May 2016imported>Labslogbot 547,458 bytes +215 l10nupdate@tin ResourceLoader cache refresh completed at Sat May 14 02:28:07 UTC 2016 (duration 8m 42s) (logmsgbot)
  • curprev 01:25, 14 May 2016imported>Labslogbot 547,243 bytes +3,056 aaron@tin Synchronized php-1.28.0-wmf.1/includes/api/ApiStashEdit.php: f33ed7ae239 (duration: 00m 36s) (logmsgbot)
  • curprev 00:53, 13 May 2016imported>Labslogbot 544,187 bytes +9,784 killing multiple icinga-wm processes on neon, running puppet. manually started and by puppet as well? (mutante)
  • curprev 00:05, 12 May 2016imported>Labslogbot 534,403 bytes +6,038 maxsem@tin Synchronized php-1.27.0-wmf.23/extensions/ZeroBanner: https://gerrit.wikimedia.org/r/#/c/288236/ (duration: 00m 25s) (logmsgbot)
  • curprev 23:59, 10 May 2016imported>Labslogbot 528,365 bytes +8,369 maxsem@tin Synchronized php-1.27.0-wmf.23/extensions/Kartographer/: Security patch (duration: 00m 25s) (logmsgbot)
  • curprev 23:15, 9 May 2016imported>Labslogbot 519,996 bytes +6,185 demon@tin Synchronized wmf-config/InitialiseSettings.php: nowiki flow beta (duration: 00m 26s) (logmsgbot)
  • curprev 07:44, 8 May 2016imported>Labslogbot 513,811 bytes +479 restarting elasticsearch server elastic1016.eqiad.wmnet (T110236) (gehel)
  • curprev 20:28, 7 May 2016imported>Labslogbot 513,332 bytes +830 deploying updated Blazegraph version for WDQS to mitigate deadlock issue (SMalyshev)
  • curprev 01:01, 7 May 2016imported>Labslogbot 512,502 bytes +2,635 ori@tin Synchronized php-1.27.0-wmf.22/includes/api/ApiStashEdit.php: If56084466: Bump PRESUME_FRESH_TTL_SEC to improve hit rate and avoid link queries (duration: 00m 34s) (logmsgbot)
  • curprev 21:37, 5 May 2016imported>Labslogbot 509,867 bytes +4,246 restarting elasticsearch server elastic1009.eqiad.wmnet (T110236) (gehel)
  • curprev 00:27, 5 May 2016imported>Labslogbot 505,621 bytes +9,792 phabricator upgrade complete (twentyafterfour)
  • curprev 00:23, 4 May 2016imported>Labslogbot 495,829 bytes +9,556 mwscript namespaceDupes.php aswikisource --merge --fix (T133505) (Dereckson)
  • curprev 00:50, 3 May 2016imported>Labslogbot 486,273 bytes +6,195 krenair@tin Synchronized wmf-config/interwiki.php: https://gerrit.wikimedia.org/r/286552 - interwiki cache update including jamwiki, horizon, and other things (duration: 00m 26s) (logmsgbot)
  • curprev 19:37, 1 May 2016imported>Labslogbot 480,078 bytes +907 enabled wdqs1002, put wdqs1001 in maintenance mode for reload (SMalyshev)
  • curprev 13:42, 30 April 2016imported>Labslogbot 479,171 bytes +770 disabled puppet on analytics1047 and scheduled downtime for the host, IO errors in the dmesg for /dev/sdd. Stopped also Hadoop daemons to remove it from the cluster temporarily (not sure how to do it properly, will write docs). (elukey)
  • curprev 01:15, 30 April 2016imported>Labslogbot 478,401 bytes +4,965 applied Ibd302e1 to terbium for debugging broken wikidata rdf dumps (aude)
  • curprev 00:27, 29 April 2016imported>Labslogbot 473,436 bytes +7,525 RT - remove libapache2-mod-php5, restart Apache, Perl apps dont need PHP (mutante)
  • curprev 00:48, 28 April 2016imported>Labslogbot 465,911 bytes +5,278 Phabricator's back online, everything seems to have gone smoothly. (twentyafterfour)
  • curprev 23:29, 26 April 2016imported>Labslogbot 460,633 bytes +6,712 catrope@tin Synchronized php-1.27.0-wmf.22/extensions/Flow: SWAT (duration: 00m 45s) (logmsgbot)
  • curprev 01:09, 26 April 2016imported>Labslogbot 453,921 bytes +7,769 reinstalling bast4001 w/ jessie (paravoid)
  • curprev 20:45, 24 April 2016imported>Labslogbot 446,152 bytes +160 ran namespaceDupes.php on aswikisource (Reedy)
  • curprev 19:09, 23 April 2016imported>Labslogbot 445,992 bytes +123 rebooted meitnerium (_joe_)
  • curprev 00:07, 23 April 2016imported>Labslogbot 445,869 bytes +1,629 deactivate phabricator account for ktr101 per global ban (Jamesofur)
  • curprev 20:34, 21 April 2016imported>Labslogbot 444,240 bytes +6,637 lowering elasticsearch high watermark to rebalance disk space across cluster (gehel)
  • curprev 01:29, 21 April 2016imported>Labslogbot 437,603 bytes +3,884 installed python-progressbar on terbium for warmup script, will be puppetized later (ori)
  • curprev 22:42, 19 April 2016imported>Labslogbot 433,719 bytes +6,908 killing rc insert query on db1065 and db1066 (ori)
  • curprev 01:11, 19 April 2016imported>Labslogbot 426,811 bytes +3,216 restbase2004 - unit cassandra-b is failed (mutante)
  • curprev 21:51, 17 April 2016imported>Labslogbot 423,595 bytes +173 Decommissioning restbase1005.eqiad.wmnet : T95253 (urandom)
  • curprev 00:10, 17 April 2016imported>Labslogbot 423,422 bytes +934 Decommissioning restbase1006.eqiad.wmnet : T95253 (urandom)
  • curprev 22:28, 15 April 2016imported>Labslogbot 422,488 bytes +2,124 rebooting bast4001 - debugging install issue (mutante)
  • curprev 23:13, 14 April 2016imported>Labslogbot 420,364 bytes +5,146 demon@tin Synchronized portals: (no message) (duration: 00m 29s) (logmsgbot)
  • curprev 01:29, 14 April 2016imported>Labslogbot 415,218 bytes +8,257 mw1211 - restart hhvm (mutante)
  • curprev 00:38, 13 April 2016imported>Labslogbot 406,961 bytes +4,751 mwscript deleteEqualMessages.php --wiki newiki (Krinkle)
  • curprev 00:37, 12 April 2016imported>Labslogbot 402,210 bytes +8,039 Deleted refreshLinksDynamic jobs for commonswiki and enwiki (T132318) (ori)
  • curprev 22:56, 10 April 2016imported>Labslogbot 394,171 bytes +521 mwscript deleteEqualMessages.php --wiki srwiki (T45917) (Krinkle)
  • curprev 20:44, 9 April 2016imported>Labslogbot 393,650 bytes +621 krenair@tin Synchronized wmf-config/LabsServices.php: sync labs-only merge of gehel's that was just left unmerged on tin (duration: 00m 41s) (logmsgbot)
  • curprev 14:03, 8 April 2016imported>Labslogbot 393,029 bytes +646 correction of the above, restbase1014-a (godog)
  • curprev 00:04, 8 April 2016imported>Labslogbot 392,383 bytes +5,554 dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Set wgSemiprotectedRestrictionLevels for en.wikipedia (Task T131976) (duration: 00m 26s) (logmsgbot)
  • curprev 23:44, 6 April 2016imported>Labslogbot 386,829 bytes +5,024 mattflaschen@tin Synchronized php-1.27.0-wmf.19/extensions/Flow/includes/Data/Listener/NotificationListener.php: Fix new topic notifications (duration: 00m 29s) (logmsgbot)
  • curprev 01:02, 6 April 2016imported>Labslogbot 381,805 bytes +3,330 krenair@tin Finished scap: https://gerrit.wikimedia.org/r/#/c/281846/ - add messages for the new extendedconfirmed protection (duration: 95m 03s) (logmsgbot)
  • curprev 01:29, 5 April 2016imported>Labslogbot 378,475 bytes +4,243 Updating CiviCRM from bfe563f2451725e734d6ddbddb3be716463a3512 to 4cc17b635eb84204cced107d5de78533cc5ce06c (eileen)
  • curprev 00:49, 4 April 2016imported>Labslogbot 374,232 bytes +1,605 krinkle@tin Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 29s) (logmsgbot)
  • curprev 20:28, 2 April 2016imported>Labslogbot 372,627 bytes +1,040 Restarted Jenkins on gallium (hashar)
  • curprev 01:04, 2 April 2016imported>Labslogbot 371,587 bytes +2,787 krinkle@tin Synchronized php-1.27.0-wmf.19/includes/specials/SpecialRedirect.php: T131328 (duration: 00m 39s) (logmsgbot)
  • curprev 00:55, 1 April 2016imported>Labslogbot 368,800 bytes +4,211 csteipp@tin Synchronized wmf-config/CommonSettings-labs.php: Synching labs revert (duration: 00m 31s) (logmsgbot)
  • curprev 00:30, 31 March 2016imported>Labslogbot 364,589 bytes +4,718 argon - removing apache and config (mutante)
  • curprev 23:55, 29 March 2016imported>Labslogbot 359,871 bytes +5,585 maxsem@tin Synchronized php-1.27.0-wmf.19/extensions/WikidataPageBanner/: https://gerrit.wikimedia.org/r/#/c/280327/ (duration: 00m 33s) (logmsgbot)
  • curprev 01:07, 29 March 2016imported>Labslogbot 354,286 bytes +3,023 restbase finish update to 3ea08751a8 (Pchelolo)
  • curprev 22:21, 27 March 2016imported>Labslogbot 351,263 bytes +746 ori@tin Synchronized wmf-config/StartProfiler.php: I1b5c620b85: Better request profiling via XWD header (duration: 00m 33s) (logmsgbot)
  • curprev 21:11, 26 March 2016imported>Labslogbot 350,517 bytes +635 removed 2FA from wikitech accounts that looked to be affected by T130892 (csteipp_ooo)
  • curprev 00:11, 26 March 2016imported>Labslogbot 349,882 bytes +2,302 scb[12]00[12] - delete changeprop main.log per request (mutante|away)
  • curprev 00:24, 25 March 2016imported>Labslogbot 347,580 bytes +6,919 catrope@tin Finished scap: Pushing out i18n changes from SWAT (duration: 26m 15s) (logmsgbot)
  • curprev 00:47, 24 March 2016imported>Labslogbot 340,661 bytes +7,363 phab upgrade/maintenance finished (twentyafterfour)
  • curprev 23:47, 22 March 2016imported>Labslogbot 333,298 bytes +7,355 maxsem@tin Synchronized php-1.27.0-wmf.18/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/279066/ (duration: 00m 27s) (logmsgbot)
  • curprev 19:58, 21 March 2016imported>Labslogbot 325,943 bytes +1,351 powercycle mw1142, console available but not ever prompting for the root password, stuck at username (akosiaris)
  • curprev 02:32, 20 March 2016imported>Labslogbot 324,592 bytes +229 l10nupdate@tin ResourceLoader cache refresh completed at Sun Mar 20 02:32:04 UTC 2016 (duration 8m 30s) (logmsgbot)
  • curprev 22:29, 19 March 2016imported>Labslogbot 324,363 bytes +708 powercycling oxygen, looks kernel-dead (jynus)
  • curprev 23:35, 18 March 2016imported>Labslogbot 323,655 bytes +5,201 krinkle@tin Synchronized php-1.27.0-wmf.17/extensions/WikimediaEvents/modules/ext.wikimediaEvents.deprecate.js: (no message) (duration: 00m 35s) (logmsgbot)
  • curprev 00:38, 18 March 2016imported>Labslogbot 318,454 bytes +10,945 reboot elastic1022.eqiad.wmnet for kernel upgrade (ebernhardson)
  • curprev 00:58, 17 March 2016imported>Labslogbot 307,509 bytes +8,165 rebooting elastic1005.eqiad.wmnet for kernel update (ebernhardson)
  • curprev 01:18, 16 March 2016imported>Labslogbot 299,344 bytes +8,885 maxsem@tin Synchronized php-1.27.0-wmf.16/includes/jobqueue/JobQueueGroup.php: Debug (duration: 00m 26s) (logmsgbot)
  • curprev 00:07, 15 March 2016imported>Labslogbot 290,459 bytes +6,164 reedy@tin Synchronized wmf-config/extension-list: Remove OAI (duration: 00m 24s) (logmsgbot)
  • curprev 02:39, 13 March 2016imported>Labslogbot 284,295 bytes +229 l10nupdate@tin ResourceLoader cache refresh completed at Sun Mar 13 02:39:06 UTC 2016 (duration 8m 35s) (logmsgbot)
  • curprev 17:12, 12 March 2016imported>Labslogbot 284,066 bytes +515 Updated operations/dumps/dcat on snapshot1003 from 92ab37d94e to e97408df39 (hoo)
  • curprev 00:55, 12 March 2016imported>Labslogbot 283,551 bytes +4,921 aaron@tin Synchronized rpc/RunJobs.php: 7a8bd37247b7dfb (duration: 00m 38s) (logmsgbot)
  • curprev 01:25, 11 March 2016imported>Labslogbot 278,630 bytes +6,403 krinkle@tin Synchronized w/static.php: 6da604f and 49c07ac (duration: 00m 39s) (logmsgbot)
  • curprev 00:45, 10 March 2016imported>Labslogbot 272,227 bytes +10,699 catrope@tin Synchronized php-1.27.0-wmf.15/extensions/ZeroBanner: SWAT (duration: 00m 28s) (logmsgbot)
  • curprev 01:03, 8 March 2016imported>Labslogbot 261,528 bytes +5,430 maxsem@tin Synchronized php-1.27.0-wmf.15/extensions/Kartographer/: https://gerrit.wikimedia.org/r/#/c/275732/ (duration: 02m 25s) (logmsgbot)
  • curprev 10:45, 6 March 2016imported>Labslogbot 256,098 bytes +260 rebooting alsafi (_joe_)
  • curprev 13:29, 5 March 2016imported>Labslogbot 255,838 bytes +429 hhvm restarted on mw1025 (ema)
  • curprev 00:01, 5 March 2016imported>Labslogbot 255,409 bytes +3,391 initiate dd replicate from labstore1001 tools snapshot to labstore1002 lv of tools-04032016 (chasemp)
  • curprev 00:51, 4 March 2016imported>Labslogbot 252,018 bytes +8,200 updated payments wiki from 111f92133cbd6b3890f53e29831ac022d3c26f51 to fbd22230ebf14e57f2c69ef1b342d4cdbc47d9a6 (ejegg)
  • curprev 01:22, 3 March 2016imported>Labslogbot 243,818 bytes +8,356 manually installed scap package on iridium, will fix in puppet immediately after maintenance is finished (twentyafterfour)
  • curprev 00:45, 2 March 2016imported>Labslogbot 235,462 bytes +7,274 krenair@tin Synchronized portals: https://gerrit.wikimedia.org/r/#/c/274316/ - try #2, this time with the submodule update (duration: 01m 17s) (logmsgbot)
  • curprev 01:03, 1 March 2016imported>Labslogbot 228,188 bytes +6,871 ori@tin Synchronized php-1.27.0-wmf.14/includes/user/User.php: I419f356b: Cache user data in memory (duration: 00m 42s) (logmsgbot)
  • curprev 22:34, 28 February 2016imported>Labslogbot 221,317 bytes +444 legoktm@tin Synchronized php-1.27.0-wmf.14/includes/parser/ParserCache.php: Include ParserCache::save() backtrace in MF pollution debug log - T124356 (duration: 00m 50s) (logmsgbot)
  • curprev 00:59, 28 February 2016imported>Labslogbot 220,873 bytes +961 Restarted logmsgbot on neon; it lost its IRC connection and did not recover it. (ori_)
  • curprev 18:41, 26 February 2016imported>Labslogbot 219,912 bytes +1,763 ori@tin Synchronized php-1.27.0-wmf.14/includes/user/User.php: I43cde3a48: Prevent duplicate memcached lookups for user record (duration: 01m 02s) (logmsgbot)
  • curprev 01:06, 26 February 2016imported>Labslogbot 218,149 bytes +5,752 catrope@tin Synchronized wmf-config/InitialiseSettings.php: Lower survey rate again (duration: 01m 05s) (logmsgbot)
  • curprev 00:57, 25 February 2016imported>Labslogbot 212,397 bytes +7,304 Started crashed Logstash process on logstash1002 (systemd doesn't restart authomatically due to T127677) (bd808)
  • curprev 01:24, 24 February 2016imported>Labslogbot 205,093 bytes +6,365 krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/272889/ (duration: 01m 35s) (logmsgbot)
  • curprev 01:28, 23 February 2016imported>Labslogbot 198,728 bytes +4,834 catrope@tin Synchronized php-1.27.0-wmf.14/extensions/VisualEditor: SWAT (duration: 01m 34s) (logmsgbot)
  • curprev 14:39, 21 February 2016imported>Labslogbot 193,894 bytes +657 restarting db2070 to test configuration change (jynus)
  • curprev 19:40, 20 February 2016imported>Labslogbot 193,237 bytes +891 ori@tin Synchronized php-1.27.0-wmf.13/extensions/MobileFrontend/includes/MobileFrontend.hooks.php: Remove live-hacked logging code for T124356 (duration: 01m 42s) (logmsgbot)
  • curprev 00:45, 20 February 2016imported>Labslogbot 192,346 bytes +4,004 ori@tin Finished scap: I2a66b40e4c6: add a dependency on xhprof/xhgui (duration: 50m 12s) (logmsgbot)
  • curprev 01:02, 19 February 2016imported>Labslogbot 188,342 bytes +7,795 restbase deploy end of fa1207e95 (mobrovac)
  • curprev 01:08, 18 February 2016imported>Labslogbot 180,547 bytes +6,223 stopped phd and started dumping phabricator's database to /srv/dumps/20160218.phabricator.sql.gz (just in case I need to roll back the update) (twentyafterfour)
  • curprev 01:27, 17 February 2016imported>Labslogbot 174,324 bytes +7,308 labservices1001 - out of disk again (T126572) - moved designate-mdns.log files to /srv/var/ (mutante)
  • curprev 21:26, 15 February 2016imported>Labslogbot 167,016 bytes +2,294 krinkle@tin Synchronized w/static.php: (no message) (duration: 00m 58s) (logmsgbot)
  • curprev 23:18, 14 February 2016imported>Labslogbot 164,722 bytes +822 repaired table and restarted replication on sanitarium (s3) (jynus)
  • curprev 06:30, 13 February 2016imported>Labslogbot 163,900 bytes +255 `nodetool stop -- COMPACTION && nodetool cleanup' on restbase1002.eqiad, an abundance of caution (https://phabricator.wikimedia.org/P2612) (urandom)
  • curprev 01:18, 13 February 2016imported>Labslogbot 163,645 bytes +2,513 omg testing this log feature that logs straight to tickets (T108720) (mutante)
  • curprev 01:26, 12 February 2016imported>Labslogbot 161,132 bytes +8,221 restbase deploy end of 6f6311f (mobrovac)
  • curprev 01:05, 11 February 2016imported>Labslogbot 152,911 bytes +8,146 krenair@mira Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/269883/1 (duration: 02m 11s) (logmsgbot)
  • curprev 01:23, 10 February 2016imported>Labslogbot 144,765 bytes +8,956 started db1048 replication. For some reason, replication was stopped. Need further investigation. (jynus)
  • curprev 01:27, 9 February 2016imported>Labslogbot 135,809 bytes +8,997 krenair@mira Synchronized php-1.27.0-wmf.12/extensions/OAuth/frontend/specialpages/SpecialMWOAuthManageConsumers.php: https://gerrit.wikimedia.org/r/#/c/269333/ (duration: 01m 19s) (logmsgbot)
  • curprev 04:54, 7 February 2016imported>Labslogbot 126,812 bytes +142 upgraded python-openstackclient python-glanceclient python-novaclient python-keystoneclient on silver (andrewbogott)
  • curprev 05:43, 6 February 2016imported>Labslogbot 126,670 bytes +100 rebooted cp2006 via racadm after crash - no crash data in logs... (bblack)
  • curprev 23:54, 5 February 2016imported>Labslogbot 126,570 bytes +4,096 nfs shaping is really writes :) (chasemp)
  • curprev 01:22, 5 February 2016imported>Labslogbot 122,474 bytes +8,046 aude@mira Started scap: Re-add user rights messages in Echo (logmsgbot)
  • curprev 01:04, 4 February 2016imported>Labslogbot 114,428 bytes +11,388 krenair@mira Synchronized php-1.27.0-wmf.12/tests: https://gerrit.wikimedia.org/r/#/c/268332/ (duration: 02m 08s) (logmsgbot)
  • curprev 23:13, 2 February 2016imported>Labslogbot 103,040 bytes +7,035 demon@mira Finished scap: everything re-sync one more time for good measure (duration: 17m 04s) (logmsgbot)
  • curprev 01:11, 2 February 2016imported>Labslogbot 96,005 bytes +4,080 ebernhardson@mira Started scap: Add Cookie statement link to footer of all WMF wikis per legal (logmsgbot)
  • curprev 00:04, 1 February 2016imported>Labslogbot 91,925 bytes +1,272 tstarling@mira Synchronized php-1.27.0-wmf.11/includes: (no message) (duration: 01m 31s) (logmsgbot)
  • curprev 23:20, 30 January 2016imported>Labslogbot 90,653 bytes +915 bd808@mira rebuilt wikiversions.php and synchronized wikiversions files: Revert all wikis to 1.27.0-wmf.10 (again) (logmsgbot)
  • curprev 00:08, 30 January 2016imported>Labslogbot 89,738 bytes +4,030 bd808@mira Synchronized php-1.27.0-wmf.11/includes/session/SessionBackend.php: Remove proposed fix for T125267 (duration: 01m 33s) (logmsgbot)
  • curprev 01:03, 29 January 2016imported>Labslogbot 85,708 bytes +8,571 krenair@mira Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/267186/ (duration: 01m 09s) (logmsgbot)
  • curprev 01:03, 28 January 2016imported>Labslogbot 77,137 bytes +4,548 krenair@mira Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264460/ (duration: 02m 30s) (logmsgbot)
  • curprev 01:29, 27 January 2016imported>Labslogbot 72,589 bytes +8,552 on terbium: ran mwscript namespaceDupes.php --wiki=wuuwiki --source-pseudo-namespace='' --add-suffix=/renamed --fix (MaxSem)
  • curprev 00:45, 26 January 2016imported>Labslogbot 64,037 bytes +6,313 mobileapps deploying c2318b6 (mobrovac)
  • curprev 02:31, 24 January 2016imported>Labslogbot 57,724 bytes +229 l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 24 02:31:21 UTC 2016 (duration 6m 58s) (logmsgbot)
  • curprev 19:03, 23 January 2016imported>Labslogbot 57,495 bytes +1,050 ebernhardson@tin Synchronized wmf-config/CirrusSearch-production.php: config change to repoint morelike search from eqiad to codfw (duration: 00m 26s) (logmsgbot)
  • curprev 01:28, 23 January 2016imported>Labslogbot 56,445 bytes +6,479 bd808@tin rebuilt wikiversions.php and synchronized wikiversions files: Temporarily back to 1.27.0-wmf11; need to rebuild l10n cache (logmsgbot)
  • curprev 01:13, 22 January 2016imported>Labslogbot 49,966 bytes +7,942 catrope@tin Started scap: Deploying OATHAuth and WikimediaMessages i18n changes (logmsgbot)
  • curprev 01:15, 21 January 2016imported>Labslogbot 42,024 bytes +7,498 Restarted logstash on logstash1003 (bd808)
  • curprev 01:27, 20 January 2016imported>Labslogbot 34,526 bytes +5,797 aaron@tin Synchronized wmf-config: Configure $wgCdnReboundPurgeDelay (duration: 00m 32s) (logmsgbot)
  • curprev 23:26, 18 January 2016imported>Labslogbot 28,729 bytes +1,261 krenair@tin Synchronized multiversion/MWMultiVersion.php: https://gerrit.wikimedia.org/r/264895 (duration: 00m 31s) (logmsgbot)
  • curprev 04:58, 17 January 2016imported>Labslogbot 27,468 bytes +449 started restbase on restbase1002 (YuviPanda)
  • curprev 19:52, 16 January 2016imported>Labslogbot 27,019 bytes +730 renaming and reimaging labcontrol2001 -> labtestweb2001 (andrewbogott)
  • curprev 22:43, 15 January 2016imported>Labslogbot 26,289 bytes −484,770 aaron@tin Synchronized wmf-config/CommonSettings.php: Set $wgCentralAuthUseSlaves for testwiki (duration: 00m 33s) (logmsgbot)
  • curprev 01:00, 15 January 2016imported>Labslogbot 511,059 bytes +2,038 krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryWatchlist.php: https://gerrit.wikimedia.org/r/#/c/264224/ (duration: 00m 31s) (logmsgbot)
  • curprev 01:28, 14 January 2016imported>Labslogbot 509,021 bytes +4,993 krenair@tin rebuilt wikiversions.php and synchronized wikiversions files: (no message) (logmsgbot)
  • curprev 01:19, 13 January 2016imported>Labslogbot 504,028 bytes +5,234 krenair@tin Synchronized php-1.27.0-wmf.10/extensions/Echo/Resources.php: https://gerrit.wikimedia.org/r/#/c/263645/ (duration: 00m 32s) (logmsgbot)
  • curprev 00:46, 12 January 2016imported>Labslogbot 498,794 bytes +5,350 krenair@tin Synchronized wmf-config/InitialiseSettings.php: rv 443026e3ad18934dd0017a258673d88104cf6b5e (duration: 00m 29s) (logmsgbot)
  • curprev 01:11, 11 January 2016imported>Labslogbot 493,444 bytes +1,147 deactivating eqiad<->GTT BGP peering, reported network issues (P2469) (paravoid)
  • curprev 18:23, 9 January 2016imported>Tim Landscheidtm 492,297 bytes +228 Reverted edits by Tegel (talk) to last revision by Labslogbot
  • curprev 23:49, 8 January 2016imported>Labslogbot 492,069 bytes +302 stalled puppet on carbon for now, messing with partman files (RobH)
  • curprev 23:24, 7 January 2016imported>Labslogbot 491,767 bytes +1,399 repooled scb1002 for mobileapps (akosiaris)
  • curprev 23:03, 6 January 2016imported>Labslogbot 490,368 bytes +361 switched restbase1009 to node 4.2 for testing, and restarted restbase; see https://phabricator.wikimedia.org/T107762 (gwicke)
  • curprev 22:38, 5 January 2016imported>Labslogbot 490,007 bytes +921 aaron@tin Synchronized rpc: 830e1ed8d80295710dc02f18102b4fadae7fca86 (duration: 00m 55s) (logmsgbot)
  • curprev 20:50, 4 January 2016imported>Labslogbot 489,086 bytes +1,072 ms-be1011 - powercycled, was frozen (mutante)
  • curprev 02:32, 3 January 2016imported>Labslogbot 488,014 bytes +228 l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 3 02:31:58 UTC 2016 (duration 6m 52s) (logmsgbot)
  • curprev 03:34, 2 January 2016imported>Labslogbot 487,786 bytes +313 deploying https://gerrit.wikimedia.org/r/261725, restarted apache2 on iridium (twentyafterfour)
  • curprev 01:04, 2 January 2016imported>Labslogbot 487,473 bytes +602 imported vagrant 1.8.1 for jessie per bd808 (YuviPanda)
  • curprev 02:31, 31 December 2015imported>Labslogbot 486,871 bytes +228 l10nupdate@tin ResourceLoader cache refresh completed at Thu Dec 31 02:31:43 UTC 2015 (duration 6m 52s) (logmsgbot)
  • curprev 15:05, 30 December 2015imported>Labslogbot 486,643 bytes +973 restarting puppet on seaborgium; openldap will restart but config should not change (andrewbogott)
  • curprev 00:08, 30 December 2015imported>Labslogbot 485,670 bytes +1,863 restarting nova-compute on labvirt1002 (andrewbogott)
  • curprev 01:02, 29 December 2015imported>Labslogbot 483,807 bytes +2,394 awight@tin Synchronized php-1.27.0-wmf.9/extensions/CentralNotice: Update CentralNotice: T122251 (duration: 00m 34s) (logmsgbot)
  • curprev 03:05, 27 December 2015imported>Labslogbot 481,413 bytes +482 run nodetool clearsnapshot -- v3 and nodetool clearsnapshot -- v1 on maps-test2001 (YuviPanda)
  • curprev 19:12, 26 December 2015imported>Labslogbot 480,931 bytes +767 restarting varnish-frontend on cp3042 (paravoid)
  • curprev 15:50, 25 December 2015imported>Labslogbot 480,164 bytes +449 testing new mariadb packages on db2070 (jynus)
  • curprev 23:28, 24 December 2015imported>Labslogbot 479,715 bytes +1,399 powercycled mw1114 (mutante)
  • curprev 01:05, 24 December 2015imported>Labslogbot 478,316 bytes +2,128 update payments from bae4d02afd8cfe1f8b8617c2f74bb36e420d281d to a7785baa7b40b442ecf0b60d47572502d0759780 (awight)
  • curprev 00:40, 23 December 2015imported>Labslogbot 476,188 bytes +3,023 krenair@tin Synchronized wmf-config/CommonSettings-labs.php: https://gerrit.wikimedia.org/r/260696 & https://gerrit.wikimedia.org/r/260699 (duration: 05m 28s) (logmsgbot)
  • curprev 00:29, 22 December 2015imported>Labslogbot 473,165 bytes +3,038 krenair@tin Synchronized php-1.27.0-wmf.9/extensions/VisualEditor: https://gerrit.wikimedia.org/r/#/c/260492/ (duration: 00m 32s) (logmsgbot)
  • curprev 23:24, 20 December 2015imported>Labslogbot 470,127 bytes +878 Katie and Jeff paged about bellatrix (Reedy)
  • curprev 21:55, 19 December 2015imported>Labslogbot 469,249 bytes +1,169 restarted zotero on sca1001, various OOM messages (_joe_)
  • curprev 01:02, 19 December 2015imported>Labslogbot 468,080 bytes +3,087 entire restbase cluster: removed 5% root reserve from data partition with tune2fs -m 0 /dev/mapper/restbase$NODE--vg-{srv,var} (gwicke)
  • curprev 01:25, 18 December 2015imported>Labslogbot 464,993 bytes +6,803 in preparation for Iaefb2d191e, disabling puppet on mc* and rdb* (ori)
  • curprev 00:28, 17 December 2015imported>Labslogbot 458,190 bytes +5,451 ori@tin Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: I552cf6b0420: Upgrade some ApiStashEdit logging calls to info() (duration: 00m 30s) (logmsgbot)
  • curprev 00:40, 16 December 2015imported>Labslogbot 452,739 bytes +7,890 catrope@tin Synchronized wmf-config/InitialiseSettings.php: Enable wmgGraphEnableGzip on all wikis (duration: 00m 30s) (logmsgbot)
  • curprev 00:34, 15 December 2015imported>Labslogbot 444,849 bytes +4,803 puppet disabled on silver since last Puppet run was at Fri Dec 11 15:27:28 UTC 2015 due to 'testing osm debugging' (Reedy)
  • curprev 11:10, 13 December 2015imported>Labslogbot 440,046 bytes +682 elastic in eqiad: restarting elastic1012 to release opened log filedesc (dcausse)
  • curprev 22:17, 12 December 2015imported>Labslogbot 439,364 bytes +1,510 updated fundraising dashboard from 961a24c9cf76966aaa7ba8c60e13b6d1d37fa859 to 59e51c4ff74c3c584daf6c5de3bb66daa764cd28 (ejegg)
  • curprev 00:30, 12 December 2015imported>Labslogbot 437,854 bytes +3,848 restarting slapd on seaborgium with manual hack (paravoid)
  • curprev 01:13, 11 December 2015imported>Labslogbot 434,006 bytes +6,887 ori@tin Synchronized php-1.27.0-wmf.8/includes/jobqueue: Ia44ec5ed4: Add per-partition JobQueueRedis aggregation + Fix bad regex in 6fe2f48df (duration: 00m 31s) (logmsgbot)
  • curprev 01:25, 10 December 2015imported>Labslogbot 427,119 bytes +7,091 ebernhardson@tin Synchronized php-1.27.0-wmf.7/extensions/CirrusSearch/: Add master timeout parameter to mapping updates so it can be increased when the master is slow (duration: 00m 30s) (logmsgbot)
  • curprev 01:19, 9 December 2015imported>Labslogbot 420,028 bytes +5,085 Updating paymentswiki from 96d8a49ecd521ac8ce75521aaf97ed8f0dbefc9a to 74143e43eb36f93be7881f626443182d3bb58cef (awight)
  • curprev 01:16, 8 December 2015imported>Labslogbot 414,943 bytes +6,034 catrope@tin Synchronized wmf-config/InitialiseSettings.php: Also blacklist Minerva for WPB (duration: 00m 28s) (logmsgbot)
  • curprev 21:48, 6 December 2015imported>Labslogbot 408,909 bytes +515 krypton unresponsive, nothing on console. shutting down, increasing instance ram from 2 to 4g, and rebooting. (ori)
  • curprev 18:30, 5 December 2015imported>Labslogbot 408,394 bytes +821 started nodetool decommission on restbase1008 (gwicke)
  • curprev 22:20, 4 December 2015imported>Labslogbot 407,573 bytes +1,808 updated dash from bed9895563fc0f5d3b96791fae6ed104b7778f0d to 9f0a4d98586f1281716e39b34d924de524458177 (ejegg)
  • curprev 00:35, 4 December 2015imported>Labslogbot 405,765 bytes +4,063 catrope@tin Synchronized php-1.27.0-wmf.7/extensions/CentralNotice: SWAT (duration: 00m 32s) (logmsgbot)
  • curprev 00:18, 3 December 2015imported>Labslogbot 401,702 bytes +2,268 Deploy paymentswiki config for T112181 (awight)
  • curprev 00:29, 2 December 2015imported>Labslogbot 399,434 bytes +1,871 puppetstoredconfigclean.rb labcontrol2001.wikimedia.org fixes icinga config (mutante)
  • curprev 00:51, 1 December 2015imported>Labslogbot 397,563 bytes +3,905 updated fundraising dash from bc0ec4bb2427ebeb2d4cd60eda65e3d82d006937 to 2ee15180150c8b685b14abdbf7c6769736fb49a0 (ejegg)
  • curprev 01:05, 30 November 2015imported>Labslogbot 393,658 bytes +1,159 bd808@tin sync-l10n completed (1.27.0-wmf.7) (duration: 01m 19s) (logmsgbot)
  • curprev 22:48, 28 November 2015imported>Labslogbot 392,499 bytes +657 bd808@tin Synchronized php-1.27.0-wmf.5/cache/l10n: bd808 testing l10nupdate sync-dir using stale branch (duration: 01m 29s) (logmsgbot)
  • curprev 21:23, 27 November 2015imported>Labslogbot 391,842 bytes +2,480 krenair@tin Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/255726/1 (duration: 00m 31s) (logmsgbot)
  • curprev 20:59, 26 November 2015imported>Labslogbot 389,362 bytes +2,051 aaron@tin Synchronized wmf-config/jobqueue-eqiad.php: Adjust connection timeout and maxPartitionsTry (duration: 00m 34s) (logmsgbot)
  • curprev 01:03, 26 November 2015imported>Labslogbot 387,311 bytes +3,058 from f867beecd3a4fb596e2d08c8d2b0e4173ad0b356 to d5afa10383cfd41adbe798c3ed2aa97839be8bc2 (awight)
  • curprev 01:03, 25 November 2015imported>Labslogbot 384,253 bytes +4,146 re-armed keyholders on mira (mutante)
  • curprev 01:00, 24 November 2015imported>Labslogbot 380,107 bytes +3,523 updated payments-wiki from ecbc49c63b259a4363c51491ddff273e3579f0e9 to 5358960d6857f589939c9b86ea37af168a4f8b0b (ejegg)
  • curprev 01:27, 23 November 2015imported>Labslogbot 376,584 bytes +291 end enwiki test election from cli to avoid confusion with real election (Jamesofur)
  • curprev 19:14, 21 November 2015imported>Labslogbot 376,293 bytes +603 updated fundraising tools from d7253eb1c6c9478ce2cf8760279d2ad1b7f0a132 to ebed29c0eccf38c812b20a957b3487a15bfa9cbc (ejegg|away)
  • curprev 00:50, 21 November 2015imported>Labslogbot 375,690 bytes +2,605 update-ubuntu-mirror on carbon completed successfully ; going to re-enable icinga-wm (ori)
  • curprev 01:29, 20 November 2015imported>Labslogbot 373,085 bytes +3,783 catrope@tin Synchronized php-1.27.0-wmf.7/extensions/Echo/includes/formatters/BasicFormatter.php: unstub wgLang to fix type hint errors (duration: 00m 20s) (logmsgbot)
  • curprev 00:29, 19 November 2015imported>Labslogbot 369,302 bytes +4,880 catrope@tin Synchronized php-1.27.0-wmf.7/extensions/Flow/: SWAT: better permissions errors for opt-in (duration: 00m 21s) (logmsgbot)
  • curprev 01:22, 18 November 2015imported>Labslogbot 364,422 bytes +4,679 rebooting mw1158 (andrewbogott)
  • curprev 00:21, 17 November 2015imported>Labslogbot 359,743 bytes +3,042 krinkle@tin Synchronized wmf-config/jobqueue-codfw.php: (no message) (duration: 00m 26s) (logmsgbot)
  • curprev 18:03, 15 November 2015imported>Labslogbot 356,701 bytes +1,175 disabling puppet, restarting mysql on es2008, es2009, es2010 for configuration check/upgrade (jynus)
  • curprev 23:31, 14 November 2015imported>Labslogbot 355,526 bytes +1,784 dsh delete php-1.26wmf22 23 and 24 (Reedy)
  • curprev 23:31, 13 November 2015imported>Labslogbot 353,742 bytes +2,055 ori@tin Synchronized rpc/RunJobs.php: I5e15ec9fb: Disable ChronologyProtector for rpc/RunJobs.php (duration: 00m 30s) (logmsgbot)
  • curprev 01:27, 13 November 2015imported>Labslogbot 351,687 bytes +4,168 bounced mw1045 HHVM for ebernhardson (YuviPanda)
  • curprev 01:02, 12 November 2015imported>Labslogbot 347,519 bytes +4,838 restbase: finished full deploy of restbase-deploy ce4abe784 (gwicke)
  • curprev 00:58, 11 November 2015imported>Labslogbot 342,681 bytes +5,075 added jenkins_1.625.1 to APT repo. precise-wikimedia (mutante)
  • curprev 00:13, 10 November 2015imported>Labslogbot 337,606 bytes +3,724 Attached Richardelainechambers@commonswiki to the global account of the same name (hoo)
  • curprev 20:43, 8 November 2015imported>Labslogbot 333,882 bytes +854 krenair@tin Synchronized php-1.27.0-wmf.5/extensions/ConfirmEdit/SimpleCaptcha/Captcha.php: https://gerrit.wikimedia.org/r/#/c/251673/ (duration: 00m 34s) (logmsgbot)
  • curprev 19:50, 7 November 2015imported>Labslogbot 333,028 bytes +1,574 krenair@tin Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: https://gerrit.wikimedia.org/r/#/c/251739/ (duration: 00m 35s) (logmsgbot)
  • curprev 00:23, 7 November 2015imported>Labslogbot 331,454 bytes +3,429 maxsem@tin Synchronized php-1.27.0-wmf.5/extensions/WikimediaMaintenance/getPageCounts.php: (no message) (duration: 00m 34s) (logmsgbot)
  • curprev 01:28, 6 November 2015imported>Labslogbot 328,025 bytes +4,445 bd808@tin Synchronized wmf-config/InitialiseSettings.php: Touch (duration: 00m 35s) (logmsgbot)
  • curprev 00:13, 5 November 2015imported>Labslogbot 323,580 bytes +5,250 updated fundraising tools from e1b60fa2c258fd4ff55905b03a4d8886132278c1 to f2f571903d32ec73aeab73ce58c6bee71717213b (ejegg)
  • curprev 01:10, 4 November 2015imported>Labslogbot 318,330 bytes +4,098 krenair@tin Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: https://gerrit.wikimedia.org/r/#/c/250874/ (duration: 00m 17s) (logmsgbot)
  • curprev 01:05, 3 November 2015imported>Labslogbot 314,232 bytes +6,334 enabled CRM module amazon_audit (awight)
  • curprev 16:59, 1 November 2015imported>Labslogbot 307,898 bytes +444 restarting cassandra on aqs1002; was out of heap space (gwicke)
  • curprev 08:57, 31 October 2015imported>Labslogbot 307,454 bytes +626 elastic1008 deleting /var/log/elasticsearch/production-search-eqiad_index_indexing_slowlog.log.[2-7] (dcausse)
  • curprev 23:53, 30 October 2015imported>Labslogbot 306,828 bytes +4,432 ori@tin Synchronized wmf-config/CommonSettings.php: Ifd3543d99: Re-enabled sidebar cache per 47eb083a0fe4 (duration: 00m 17s) (logmsgbot)
  • curprev 00:48, 30 October 2015imported>Labslogbot 302,396 bytes +8,055 tgr@tin Synchronized php-1.27.0-wmf.4/includes/specials/SpecialUserlogin.php: T117027 (duration: 00m 18s) (logmsgbot)
  • curprev 01:06, 29 October 2015imported>Labslogbot 294,341 bytes +5,787 lowered gc_grace on wikipedia parsoid html and data-parsoid keyspaces to 24 hours (gwicke)
  • curprev 00:55, 28 October 2015imported>Labslogbot 288,554 bytes +5,206 deployed graphoid - https://gerrit.wikimedia.org/r/#/c/249324/ (yurik)
  • curprev 01:02, 27 October 2015imported>Labslogbot 283,348 bytes +4,898 krinkle@tin Synchronized php-1.27.0-wmf.3/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: T116693 (duration: 00m 19s) (logmsgbot)
  • curprev 05:36, 25 October 2015imported>Labslogbot 278,450 bytes +373 l10nupdate@tin ResourceLoader cache refresh completed at Sun Oct 25 05:36:33 UTC 2015 (duration 36m 32s) (logmsgbot)
  • curprev 19:34, 24 October 2015imported>Labslogbot 278,077 bytes +707 deployed https://gerrit.wikimedia.org/r/#/c/248638/ and restarted apache on iridium (twentyafterfour)
  • curprev 00:21, 24 October 2015imported>Labslogbot 277,370 bytes +3,609 started second machine (nobelium) performing copy of elasticsearch indices to codfw with 40 threads (ebernhardson)
  • curprev 00:10, 23 October 2015imported>Labslogbot 273,761 bytes +5,419 krenair@tin Synchronized w/static/images/project-logos/anwiki.png: https://gerrit.wikimedia.org/r/#/c/247253/ (duration: 00m 16s) (logmsgbot)
  • curprev 01:10, 22 October 2015imported>Labslogbot 268,342 bytes +6,010 krenair@mira Synchronized README: testing sync from mira (duration: 00m 17s) (logmsgbot)
  • curprev 01:12, 21 October 2015imported>Labslogbot 262,332 bytes +7,026 hoo@tin Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 17s) (logmsgbot)
  • curprev 23:43, 19 October 2015imported>Labslogbot 255,306 bytes +4,671 ebernhardson@tin Synchronized wmf-config/InitialiseSettings.php: testwiki Graphoid to restbase (duration: 00m 17s) (logmsgbot)
  • curprev 10:04, 18 October 2015imported>Labslogbot 250,635 bytes +994 uploaded to apt.wikimedia.org trusty-wikimedia: apertium-mlt-ara_0.1.0~r57554-1 (akosiaris)
  • curprev 20:02, 17 October 2015imported>Labslogbot 249,641 bytes +751 powercycle analytics1034, no console no ssh (godog)
  • curprev 20:52, 16 October 2015imported>Labslogbot 248,890 bytes +1,454 Restarting Jenkins to remove potential dead locks before the week-end (hashar)
  • curprev 22:09, 15 October 2015imported>Labslogbot 247,436 bytes +1,474 don't worry, _joe_ was around and we approved Roan's last deploy as an exception :) (greg-g)
  • curprev 23:46, 14 October 2015imported>Labslogbot 245,962 bytes +3,718 demon@tin Synchronized wmf-config/CommonSettings.php: Nevermind on skipping timed text, extension config is weird (duration: 00m 17s) (logmsgbot)
  • curprev 00:19, 14 October 2015imported>Labslogbot 242,244 bytes +4,657 updated civicrm from c7af7634e75eb8702f5e16081a86ab86ce69c7c2 to ba39f3181431c8409416e580dcf6e15ac5f96a21 (ejegg)
  • curprev 00:51, 13 October 2015imported>Labslogbot 237,587 bytes +4,478 catrope@tin Synchronized wmf-config/InitialiseSettings.php: Reapply: Flow-occupy talk namespaces on sewikimedia (duration: 01m 13s) (logmsgbot)
  • curprev 04:57, 11 October 2015imported>Labslogbot 233,109 bytes +373 l10nupdate@tin ResourceLoader cache refresh completed at Sun Oct 11 04:57:40 UTC 2015 (duration 57m 39s) (logmsgbot)
  • curprev 14:51, 10 October 2015imported>Southparkfan 232,736 bytes −666,193 Archive SAL
  • curprev 22:20, 9 October 2015imported>Labslogbot 898,929 bytes +1,949 took down Fundraising campaigns (ejegg)
  • curprev 01:24, 9 October 2015imported>Labslogbot 896,980 bytes +3,359 catrope@tin Synchronized php-1.27.0-wmf.2/extensions/CentralNotice/resources/subscribing/ext.centralNotice.display.js: Add period to try to flush out ResourceLoader issue (duration: 01m 26s) (logmsgbot)
  • curprev 01:17, 8 October 2015imported>Labslogbot 893,621 bytes +5,436 phabricator is now up and running again (twentyafterfour)
  • curprev 00:05, 7 October 2015imported>Labslogbot 888,185 bytes +2,731 catrope@tin Synchronized php-1.27.0-wmf.2/extensions/Echo: SWAT (duration: 00m 17s) (logmsgbot)
  • curprev 23:53, 5 October 2015imported>Labslogbot 885,454 bytes +2,642 krenair@tin Synchronized php-1.27.0-wmf.1/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/243833/ (duration: 00m 17s) (logmsgbot)
  • curprev 23:36, 4 October 2015imported>Labslogbot 882,812 bytes +580 ori@tin Synchronized php-1.27.0-wmf.1/extensions/ContentTranslation/extension.json: 8c80ec1273: Updated mediawiki/core Project: mediawiki/extensions/ContentTranslation (duration: 00m 17s) (logmsgbot)
  • curprev 15:33, 3 October 2015imported>Labslogbot 882,232 bytes +706 stopping temporarily labsdb1004 mariadb to complete clone process (jynus)
  • curprev 22:46, 2 October 2015imported>Labslogbot 881,526 bytes +2,456 holmium: apt-get clean for a little disk space - /var/log/designate/designate-mdns.log is more than half the size of / - needs logrotate (mutante)
  • curprev 00:44, 2 October 2015imported>Labslogbot 879,070 bytes +5,530 ori@tin Synchronized php-1.27.0-wmf.1/includes/resourceloader: I21bb3f08e7f and follow-ups (duration: 00m 18s) (logmsgbot)
  • curprev 22:55, 30 September 2015imported>Labslogbot 873,540 bytes +2,561 twentyafterfour@tin Synchronized php-1.27.0-wmf.1/: deploying several fixes to the branch (duration: 01m 52s) (logmsgbot)
  • curprev 01:02, 30 September 2015imported>Labslogbot 870,979 bytes +4,769 ori@tin Synchronized php-1.26wmf24/vendor: 940124a7db: Updated mediawiki/core Project: mediawiki/vendor ff5e254f7eddf811f6f66b4a4063b1a8cc70f265 (duration: 00m 21s) (logmsgbot)
  • curprev 01:13, 29 September 2015imported>Labslogbot 866,210 bytes +2,479 didnt powercycle analytics1035 yet, it recovered on its own. (robh)
  • curprev 23:17, 27 September 2015imported>Labslogbot 863,731 bytes +851 ori@tin Synchronized php-1.26wmf24/./resources/src/mediawiki.toolbar/toolbar.less: I94ced06178: mediawiki.toolbar: temporary workaround for T113868 (duration: 00m 17s) (logmsgbot)
  • curprev 19:04, 26 September 2015imported>Labslogbot 862,880 bytes +189 restarting Jenkins. Just in case :-D (hashar)
  • curprev 23:54, 25 September 2015imported>Labslogbot 862,691 bytes +2,772 ori@tin Synchronized php-1.26wmf24/includes/GlobalFunctions.php: 37c6972f94: Made wfIsBadImage() use APC (duration: 00m 17s) (logmsgbot)
  • curprev 01:28, 25 September 2015imported>Labslogbot 859,919 bytes +5,372 krinkle@tin Synchronized php-1.26wmf24/extensions/WikimediaEvents: I5608f8ffd1c - Fix trailing comma (duration: 00m 17s) (logmsgbot)
  • curprev 01:15, 24 September 2015imported>Labslogbot 854,547 bytes +4,662 ori@tin Synchronized php-1.26wmf24/includes: Ifa0d4cfe8e3: Backport I1ff61153d and I8e4c3d5a5 (duration: 00m 23s) (logmsgbot)
  • curprev 00:56, 23 September 2015imported>Labslogbot 849,885 bytes +4,192 legoktm@tin Synchronized php-1.26wmf24/extensions/Echo/Hooks.php: Remove duplicate 'MediaWiki' prefix from echo.unseen stats (duration: 00m 12s) (logmsgbot)
  • curprev 00:32, 22 September 2015imported>Labslogbot 845,693 bytes +4,741 Disabled Puppet for 24h on hafnium and stopped ganglia-monitor. gmond was saturating CPU. (ori)
  • curprev 22:34, 20 September 2015imported>Labslogbot 840,952 bytes +735 reloda pybal on lvs1012 (yuvipanda)
  • curprev 23:12, 19 September 2015imported>Labslogbot 840,217 bytes +1,606 begining Cassandra repair on restbase1005 (nodetool repair -pr) (urandom)
  • curprev 23:22, 18 September 2015imported>Labslogbot 838,611 bytes +4,102 update fundraising-tools from 3e0e3ae799a507b378d0ece3e71631b10b361329 to e1b60fa2c258fd4ff55905b03a4d8886132278c1 (awight)
  • curprev 00:14, 18 September 2015imported>Labslogbot 834,509 bytes +6,909 restarted logstash on logstash1001 (ori)
  • curprev 00:35, 17 September 2015imported>Labslogbot 827,600 bytes +6,029 updated payments from 155cdeb737c01baf62551292764fd2f5a93a9a63 to 1bdd287b083032ff418434ad6bb6920735af918a (cwdent)
  • curprev 01:03, 16 September 2015imported>Labslogbot 821,571 bytes +6,779 krinkle@tin Synchronized php-1.26wmf23/resources/src/mediawiki/mediawiki.js: hotfix Ia2fcd13f4 (duration: 00m 12s) (logmsgbot)
  • curprev 00:50, 15 September 2015imported>Labslogbot 814,792 bytes +4,143 reinstalling lvs300[34] to jessie (bblack)
  • curprev 06:02, 13 September 2015imported>Labslogbot 810,649 bytes +363 l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 13 06:02:52 UTC 2015 (duration 2m 51s) (logmsgbot)
  • curprev 20:15, 12 September 2015imported>Labslogbot 810,286 bytes +677 Rolling back Echo to 1.26wmf21 branch on mw1017 (testwiki) to measure increase in render-blocking CSS size (ori)
  • curprev 21:21, 11 September 2015imported>Labslogbot 809,609 bytes +1,756 shutdown nodepool on labnodepool1001.eqiad.wmnet until monday (hashar)
  • curprev 01:16, 11 September 2015imported>Labslogbot 807,853 bytes +4,964 ori@tin Synchronized php-1.26wmf22/extensions/TitleBlacklist: 9bf13dbe0b, 3203b045f7 (duration: 00m 12s) (logmsgbot)
  • curprev 01:04, 10 September 2015imported>Labslogbot 802,889 bytes +5,413 ori@tin Synchronized php-1.26wmf21/extensions/NavigationTiming: I2605c746b: Ensure timings are reported after the page has loaded (duration: 00m 13s) (logmsgbot)
  • curprev 00:00, 9 September 2015imported>Labslogbot 797,476 bytes +4,248 catrope@tin Finished scap: Need to update i18n for a new Echo message (duration: 23m 08s) (logmsgbot)
  • curprev 00:46, 8 September 2015imported>Labslogbot 793,228 bytes +1,579 mwscript deleteEqualMessages.php --wiki eswiki (Krinkle)
  • curprev 04:28, 6 September 2015imported>Labslogbot 791,649 bytes +364 l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 6 04:27:57 UTC 2015 (duration 27m 56s) (logmsgbot)
  • curprev 23:37, 5 September 2015imported>Labslogbot 791,285 bytes +434 mwscript deleteEqualMessages.php --wiki fywiktionary (Krinkle)
  • curprev 23:52, 4 September 2015imported>Labslogbot 790,851 bytes +2,618 mattflaschen@tin Synchronized wmf-config/InitialiseSettings-labs.php: Beta-only change (duration: 00m 12s) (logmsgbot)
  • curprev 00:11, 4 September 2015imported>Labslogbot 788,233 bytes +6,425 krinkle@tin Synchronized php-1.26wmf21/includes/resourceloader/ResourceLoader.php: I24f68e34a9fa4918 (duration: 00m 12s) (logmsgbot)
  • curprev 00:36, 3 September 2015imported>Labslogbot 781,808 bytes +5,582 ori@tin Synchronized php-1.26wmf21/includes/parser/Preprocessor_Hash.php: Idd1acd903: Decline to cache preprocessor items larger than 1 Mb (duration: 00m 11s) (logmsgbot)
  • curprev 00:33, 2 September 2015imported>Labslogbot 776,226 bytes +4,769 krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/235366/ (duration: 00m 13s) (logmsgbot)
  • curprev 23:56, 31 August 2015imported>Labslogbot 771,457 bytes +4,638 krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/233665/ (duration: 00m 11s) (logmsgbot)
  • curprev 12:58, 30 August 2015imported>Labslogbot 766,819 bytes +1,020 lvchange -ay labstore/others on labstore1002 (godog)
  • curprev 15:26, 29 August 2015imported>Labslogbot 765,799 bytes +863 killing idle mysql connections from phabricator and setting wait and interactive timeout to 60 (jynus)
  • curprev 23:45, 28 August 2015imported>Labslogbot 764,936 bytes +2,516 krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/234679/ (duration: 06m 56s) (logmsgbot)
  • curprev 00:24, 28 August 2015imported>Labslogbot 762,420 bytes +5,788 powercycled mw2027 (mutante)
  • curprev 00:31, 27 August 2015imported>Labslogbot 756,632 bytes +6,046 finished phabricator upgrade, everything appears to be working (twentyafterfour)
  • curprev 00:28, 26 August 2015imported>Labslogbot 750,586 bytes +5,150 ori@tin Synchronized php-1.26wmf20/extensions/Scribunto/engines/LuaCommon/LuaCommon.php: (no message) (duration: 00m 13s) (logmsgbot)
  • curprev 23:46, 24 August 2015imported>Labslogbot 745,436 bytes +3,386 mattflaschen@tin Synchronized wmf-config: Remove wgFlowOccupyPages (duration: 00m 12s) (logmsgbot)
  • curprev 16:54, 23 August 2015imported>Labslogbot 742,050 bytes +219 bouncing Cassandra on restbase1001 to apply temporary GC settings (urandom)
  • curprev 23:08, 22 August 2015imported>Labslogbot 741,831 bytes +953 krenair@tin Synchronized php-1.26wmf19/extensions/AbuseFilter/maintenance/addMissingLoggingEntries.php: (no message) (duration: 01m 05s) (logmsgbot)
  • curprev 00:26, 22 August 2015imported>Labslogbot 740,878 bytes +1,894 deleting blog.sh and blog_pageviews crontab from stat1003 (mutante)
  • curprev 23:40, 20 August 2015imported>Labslogbot 738,984 bytes +4,846 ebernhardson@tin Synchronized php-1.26wmf19/extensions/CirrusSearch/: Fix some cirrussearch logspam (duration: 00m 13s) (logmsgbot)
  • curprev 21:20, 19 August 2015imported>Labslogbot 734,138 bytes +4,027 livehack reverted sodium back to normal, testing done (robh)
  • curprev 01:11, 19 August 2015imported>Labslogbot 730,111 bytes +3,199 updated kartotherian (yurik)
  • curprev 00:02, 18 August 2015imported>Labslogbot 726,912 bytes +3,254 analytics1041 down, attempting power cycle (ottomata)
  • curprev 18:15, 16 August 2015imported>Labslogbot 723,658 bytes +694 krenair@tin Synchronized wmf-config/extension-list: https://gerrit.wikimedia.org/r/#/c/169612/ - remove Extension:Oversight (duration: 00m 21s) (logmsgbot)
  • curprev 22:56, 15 August 2015imported>Labslogbot 722,964 bytes +961 removed 13 bounce_records for User:odder from bouncehandler database (legoktm)
  • curprev 01:15, 15 August 2015imported>Labslogbot 722,003 bytes +2,802 starting broker on analytics1012, camus wasn't happy about that either. hrm. (ottomata)
  • curprev 23:45, 13 August 2015imported>Labslogbot 719,201 bytes +6,324 rollback paymentswiki from 2e7b449224317779d53ff84527166c0d378a0a40 to 823393264d6795bbaec490ff86f17580f722e598 (awight)
  • curprev 01:26, 13 August 2015imported>Labslogbot 712,877 bytes +3,637 Restarted conversion of support desk from LQT->Flow using convertLqtPageOnLocalWiki.php, using hhvm on mw1041 (matt_flaschen)
  • curprev 01:22, 12 August 2015imported>Labslogbot 709,240 bytes +5,841 zirconium - shut down, i'm sure, mollyguard (mutante)
  • curprev 01:20, 11 August 2015imported>Labslogbot 703,399 bytes +4,246 ori@tin Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoader.php: I2089b21fc: Revert resourceloader: Add must-revalidate to Cache-Control (duration: 00m 12s) (logmsgbot)
  • curprev 17:57, 9 August 2015imported>Labslogbot 699,153 bytes +783 issuing nodetool cleanup on restbase1006 (urandom)
  • curprev 14:58, 8 August 2015imported>Labslogbot 698,370 bytes +1,003 issuing nodetool cleanup on restbase1005 (urandom)
  • curprev 21:00, 7 August 2015imported>Labslogbot 697,367 bytes +2,551 krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/230227/ - should be a noop for prod (duration: 00m 12s) (logmsgbot)
  • curprev 01:08, 7 August 2015imported>Labslogbot 694,816 bytes +7,694 ori Synchronized php-1.26wmf17/extensions/Gadgets: I2089b21fc: Updated mediawiki/core Project: mediawiki/extensions/Gadgets (duration: 00m 13s) (logmsgbot)
  • curprev 00:49, 6 August 2015imported>Labslogbot 687,122 bytes +7,735 reset password for User:Tonval after identify verification (Jamesofur)
  • curprev 23:59, 4 August 2015imported>Labslogbot 679,387 bytes +4,864 maxsem Synchronized php-1.26wmf16/extensions/WikimediaEvents/: SWAT (duration: 00m 12s) (logmsgbot)
  • curprev 00:02, 4 August 2015imported>Labslogbot 674,523 bytes +6,039 updated paymentswiki to a8c0ecbedef6179c78ed833da9f2049cb0f2641b (awight)
  • curprev 00:03, 3 August 2015imported>Labslogbot 668,484 bytes +789 krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/228198/ (duration: 00m 12s) (logmsgbot)
  • curprev 06:05, 1 August 2015imported>Labslogbot 667,695 bytes +491 removing some old apache access logs from mw1114 (_joe_)
  • curprev 00:12, 1 August 2015imported>Labslogbot 667,204 bytes +3,845 ori Synchronized extract2.php: Ie919881a4: Add an API listing template to the allowed templates in extract2.php (logmsgbot)
  • curprev 00:35, 31 July 2015imported>Labslogbot 663,359 bytes +7,505 catrope Synchronized php-1.26wmf16/extensions/Flow/includes/Model/WikiReference.php: debugging (duration: 00m 12s) (logmsgbot)
  • curprev 00:48, 30 July 2015imported>Labslogbot 655,854 bytes +7,841 ori Synchronized php-1.26wmf15/includes/Message.php: 160f69871c: Debug logging for T102199 (duration: 00m 13s) (logmsgbot)
  • curprev 00:43, 29 July 2015imported>Labslogbot 648,013 bytes +4,914 ori Synchronized php-1.26wmf15/extensions/AbuseFilter: Revert "Revert "Conversion to using getMainStashInstance()"" (duration: 00m 12s) (logmsgbot)
  • curprev 01:11, 28 July 2015imported>Labslogbot 643,099 bytes +5,843 krenair Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/227371/ (duration: 00m 11s) (logmsgbot)
  • curprev 01:18, 27 July 2015imported>Labslogbot 637,256 bytes +1,228 Re-pooling mw1159 and mw1160; ran out of time for debugging. (ori)
  • curprev 20:51, 25 July 2015imported>Labslogbot 636,028 bytes +1,141 rolling restart of restbase instances (gwicke)
  • curprev 21:57, 24 July 2015imported>Labslogbot 634,887 bytes +2,829 running mwscript populateContentModel.php --wiki=enwiki --ns=all --table=page (legoktm)
  • curprev 00:21, 24 July 2015imported>Labslogbot 632,058 bytes +5,446 Re-enabled Puppet on mw1153 (ori)
  • curprev 01:05, 23 July 2015imported>Labslogbot 626,612 bytes +6,733 phab is back (twentyafterfour)
  • curprev 23:45, 21 July 2015imported>Labslogbot 619,879 bytes +4,786 catrope Synchronized wmf-config/InitialiseSettings.php: Set $wgVectorResponsive = true on testwiki (duration: 00m 12s) (logmsgbot)
  • curprev 23:44, 20 July 2015imported>Labslogbot 615,093 bytes +1,285 removed experimental nodes (1008, 1009) from system.peers on production C* nodes (gwicke)
  • curprev 00:02, 20 July 2015imported>Labslogbot 613,808 bytes +1,946 DNS update - adding language "azb" to langlist (mutante)
  • curprev 20:58, 18 July 2015imported>Labslogbot 611,862 bytes +821 legoktm Synchronized wmf-config/InitialiseSettings-labs.php: labs only (duration: 00m 12s) (logmsgbot)
  • curprev 00:49, 18 July 2015imported>Labslogbot 611,041 bytes +2,129 restored recurring globalcollect batch size of 250 (ejegg)
  • curprev 21:27, 16 July 2015imported>Labslogbot 608,912 bytes +3,677 bounced nutcracker on mw1139 as well. hashar noticed flood of errors from these hosts on https://logstash.wikimedia.org/#/dashboard/elasticsearch/mediawiki-errors . lack of monitoring / alerts is troubling. (ori)
  • curprev 01:22, 16 July 2015imported>Labslogbot 605,235 bytes +7,251 es1.6 upgrade: upgrade elastic1023 (manybubbles)
  • curprev 23:46, 14 July 2015imported>Labslogbot 597,984 bytes +3,095 es1.6 upgrade: upgraded elastic1011 (manybubbles)
  • curprev 01:22, 14 July 2015imported>Labslogbot 594,889 bytes +5,544 springle Synchronized wmf-config/db-eqiad.php: repool db1037; depool db1030 (duration: 00m 13s) (logmsgbot)
  • curprev 14:59, 12 July 2015imported>Labslogbot 589,345 bytes +1,013 upgraded most packages on sodium (bblack)
  • curprev 19:48, 11 July 2015imported>Labslogbot 588,332 bytes +1,475 stopping labsdb1002 after table corruption has been detected (jynus)
  • curprev 00:46, 11 July 2015imported>Labslogbot 586,857 bytes +2,703 Upgraded Elasticsearch to 1.6.0 on logstash1005; replicas recovering now (bd808)
  • curprev 00:39, 10 July 2015imported>Labslogbot 584,154 bytes +5,464 starting restbase1004 (urandom)
  • curprev 01:07, 9 July 2015imported>Labslogbot 578,690 bytes +4,522 uranium - deleted apache logs older than 90 days (mutante)
  • curprev 23:54, 7 July 2015imported>Labslogbot 574,168 bytes +2,953 kafka brokers 1018 & 1021 were demoted; i have triggered a leader election and they are leaders again (jgage)
  • curprev 01:12, 7 July 2015imported>Labslogbot 571,215 bytes +5,225 Re-pooled mw1152 at 20:46 UTC, did not log it then. (ori)
  • curprev 22:30, 5 July 2015imported>Labslogbot 565,990 bytes +1,663 Restarted logstash on logstah1001; Hung due to OOM errors (bd808)
  • curprev 23:49, 4 July 2015imported>Labslogbot 564,327 bytes +985 Ran "mwscript updateSpecialPages.php labswiki --override --only=Wantedpages" on silver, completed in 0.44 seconds (Krenair)
  • curprev 01:00, 4 July 2015imported>Labslogbot 563,342 bytes +6,467 reload haproxy dbproxy1004 (springle)
  • curprev 22:34, 2 July 2015imported>Labslogbot 556,875 bytes +3,571 legoktm Synchronized php-1.26wmf12/extensions/CentralAuth/: Made use of new USE_MULTI_COMMIT flag in user merge jobs (duration: 00m 18s) (logmsgbot)
  • curprev 00:44, 2 July 2015imported>Labslogbot 553,304 bytes +3,706 Repooling mw1152 (HHVM image scaler) for testing) (ori)
  • curprev 01:03, 1 July 2015imported>Labslogbot 549,598 bytes +6,042 Disabling Puppet on mw1152 for 12h to hack apache config to log locally (ori)
  • curprev 00:37, 30 June 2015imported>Labslogbot 543,556 bytes +4,176 restbase1* upgrade to cassandra 2.1.7 completed (godog)
  • curprev 00:52, 29 June 2015imported>Labslogbot 539,380 bytes +718 restart eventlogging auto-purge on m4 (springle)
  • curprev 23:30, 27 June 2015imported>Labslogbot 538,662 bytes +510 Deleted corrupt shards on logstash1004 and logstash1005. Recovery in process (bd808)
  • curprev 23:57, 26 June 2015imported>Labslogbot 538,152 bytes +2,020 Logstash log ingestion working again after forcing recovery of replicas for logstash-2015.06.26; new logs were being rejected with only a primary shard available (bd808)
  • curprev 00:51, 26 June 2015imported>Labslogbot 536,132 bytes +6,360 reverted restbase1001 canary to 90817c2a (gwicke)
  • curprev 00:43, 25 June 2015imported>Labslogbot 529,772 bytes +4,757 update crm from bd8a00196071ddd04efbff7b30567dd9357c9000 to e923225e423948bd70440e2d1131460b10cefac1 (awight)
  • curprev 01:01, 24 June 2015imported>Labslogbot 525,015 bytes +7,464 rolling restart of cassandra instances to rule out a single node in funky state causing elevated p99 latency (gwicke)
  • curprev 01:17, 23 June 2015imported>Labslogbot 517,551 bytes +2,806 krinkle Synchronized docroot and w: (no message) (duration: 00m 12s) (logmsgbot)
  • curprev 16:26, 22 June 2015imported>Labslogbot 514,745 bytes +514,745 jynus Synchronized wmf-config/db-eqiad.php: Repool es1001, depool es1002 (duration: 00m 14s) (logmsgbot)
(newest | oldest) View ( | older 500) (20 | 50 | 100 | 250 | 500)