Server Admin Log: Revision history

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

curprev 01:0601:06, 26 February 2016‎ imported>Labslogbot‎ 218,149 bytes +5,752‎ catrope@tin Synchronized wmf-config/InitialiseSettings.php: Lower survey rate again (duration: 01m 05s) (logmsgbot)

curprev 00:5700:57, 25 February 2016‎ imported>Labslogbot‎ 212,397 bytes +7,304‎ Started crashed Logstash process on logstash1002 (systemd doesn't restart authomatically due to T127677) (bd808)

curprev 01:2401:24, 24 February 2016‎ imported>Labslogbot‎ 205,093 bytes +6,365‎ krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/272889/ (duration: 01m 35s) (logmsgbot)

curprev 01:2801:28, 23 February 2016‎ imported>Labslogbot‎ 198,728 bytes +4,834‎ catrope@tin Synchronized php-1.27.0-wmf.14/extensions/VisualEditor: SWAT (duration: 01m 34s) (logmsgbot)

curprev 14:3914:39, 21 February 2016‎ imported>Labslogbot‎ 193,894 bytes +657‎ restarting db2070 to test configuration change (jynus)

curprev 19:4019:40, 20 February 2016‎ imported>Labslogbot‎ 193,237 bytes +891‎ ori@tin Synchronized php-1.27.0-wmf.13/extensions/MobileFrontend/includes/MobileFrontend.hooks.php: Remove live-hacked logging code for T124356 (duration: 01m 42s) (logmsgbot)
curprev 00:4500:45, 20 February 2016‎ imported>Labslogbot‎ 192,346 bytes +4,004‎ ori@tin Finished scap: I2a66b40e4c6: add a dependency on xhprof/xhgui (duration: 50m 12s) (logmsgbot)

curprev 01:0201:02, 19 February 2016‎ imported>Labslogbot‎ 188,342 bytes +7,795‎ restbase deploy end of fa1207e95 (mobrovac)

curprev 01:0801:08, 18 February 2016‎ imported>Labslogbot‎ 180,547 bytes +6,223‎ stopped phd and started dumping phabricator's database to /srv/dumps/20160218.phabricator.sql.gz (just in case I need to roll back the update) (twentyafterfour)

curprev 01:2701:27, 17 February 2016‎ imported>Labslogbot‎ 174,324 bytes +7,308‎ labservices1001 - out of disk again (T126572) - moved designate-mdns.log files to /srv/var/ (mutante)

curprev 21:2621:26, 15 February 2016‎ imported>Labslogbot‎ 167,016 bytes +2,294‎ krinkle@tin Synchronized w/static.php: (no message) (duration: 00m 58s) (logmsgbot)

curprev 23:1823:18, 14 February 2016‎ imported>Labslogbot‎ 164,722 bytes +822‎ repaired table and restarted replication on sanitarium (s3) (jynus)

curprev 06:3006:30, 13 February 2016‎ imported>Labslogbot‎ 163,900 bytes +255‎ `nodetool stop -- COMPACTION && nodetool cleanup' on restbase1002.eqiad, an abundance of caution (https://phabricator.wikimedia.org/P2612) (urandom)
curprev 01:1801:18, 13 February 2016‎ imported>Labslogbot‎ 163,645 bytes +2,513‎ omg testing this log feature that logs straight to tickets (T108720) (mutante)

curprev 01:2601:26, 12 February 2016‎ imported>Labslogbot‎ 161,132 bytes +8,221‎ restbase deploy end of 6f6311f (mobrovac)

curprev 01:0501:05, 11 February 2016‎ imported>Labslogbot‎ 152,911 bytes +8,146‎ krenair@mira Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/269883/1 (duration: 02m 11s) (logmsgbot)

curprev 01:2301:23, 10 February 2016‎ imported>Labslogbot‎ 144,765 bytes +8,956‎ started db1048 replication. For some reason, replication was stopped. Need further investigation. (jynus)

curprev 01:2701:27, 9 February 2016‎ imported>Labslogbot‎ 135,809 bytes +8,997‎ krenair@mira Synchronized php-1.27.0-wmf.12/extensions/OAuth/frontend/specialpages/SpecialMWOAuthManageConsumers.php: https://gerrit.wikimedia.org/r/#/c/269333/ (duration: 01m 19s) (logmsgbot)

curprev 04:5404:54, 7 February 2016‎ imported>Labslogbot‎ 126,812 bytes +142‎ upgraded python-openstackclient python-glanceclient python-novaclient python-keystoneclient on silver (andrewbogott)

curprev 05:4305:43, 6 February 2016‎ imported>Labslogbot‎ 126,670 bytes +100‎ rebooted cp2006 via racadm after crash - no crash data in logs... (bblack)

curprev 23:5423:54, 5 February 2016‎ imported>Labslogbot‎ 126,570 bytes +4,096‎ nfs shaping is really writes :) (chasemp)
curprev 01:2201:22, 5 February 2016‎ imported>Labslogbot‎ 122,474 bytes +8,046‎ aude@mira Started scap: Re-add user rights messages in Echo (logmsgbot)

curprev 01:0401:04, 4 February 2016‎ imported>Labslogbot‎ 114,428 bytes +11,388‎ krenair@mira Synchronized php-1.27.0-wmf.12/tests: https://gerrit.wikimedia.org/r/#/c/268332/ (duration: 02m 08s) (logmsgbot)

curprev 23:1323:13, 2 February 2016‎ imported>Labslogbot‎ 103,040 bytes +7,035‎ demon@mira Finished scap: everything re-sync one more time for good measure (duration: 17m 04s) (logmsgbot)
curprev 01:1101:11, 2 February 2016‎ imported>Labslogbot‎ 96,005 bytes +4,080‎ ebernhardson@mira Started scap: Add Cookie statement link to footer of all WMF wikis per legal (logmsgbot)

curprev 00:0400:04, 1 February 2016‎ imported>Labslogbot‎ 91,925 bytes +1,272‎ tstarling@mira Synchronized php-1.27.0-wmf.11/includes: (no message) (duration: 01m 31s) (logmsgbot)

curprev 23:2023:20, 30 January 2016‎ imported>Labslogbot‎ 90,653 bytes +915‎ bd808@mira rebuilt wikiversions.php and synchronized wikiversions files: Revert all wikis to 1.27.0-wmf.10 (again) (logmsgbot)
curprev 00:0800:08, 30 January 2016‎ imported>Labslogbot‎ 89,738 bytes +4,030‎ bd808@mira Synchronized php-1.27.0-wmf.11/includes/session/SessionBackend.php: Remove proposed fix for T125267 (duration: 01m 33s) (logmsgbot)

curprev 01:0301:03, 29 January 2016‎ imported>Labslogbot‎ 85,708 bytes +8,571‎ krenair@mira Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/267186/ (duration: 01m 09s) (logmsgbot)

curprev 01:0301:03, 28 January 2016‎ imported>Labslogbot‎ 77,137 bytes +4,548‎ krenair@mira Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264460/ (duration: 02m 30s) (logmsgbot)

curprev 01:2901:29, 27 January 2016‎ imported>Labslogbot‎ 72,589 bytes +8,552‎ on terbium: ran mwscript namespaceDupes.php --wiki=wuuwiki --source-pseudo-namespace='' --add-suffix=/renamed --fix (MaxSem)

curprev 00:4500:45, 26 January 2016‎ imported>Labslogbot‎ 64,037 bytes +6,313‎ mobileapps deploying c2318b6 (mobrovac)

curprev 02:3102:31, 24 January 2016‎ imported>Labslogbot‎ 57,724 bytes +229‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 24 02:31:21 UTC 2016 (duration 6m 58s) (logmsgbot)

curprev 19:0319:03, 23 January 2016‎ imported>Labslogbot‎ 57,495 bytes +1,050‎ ebernhardson@tin Synchronized wmf-config/CirrusSearch-production.php: config change to repoint morelike search from eqiad to codfw (duration: 00m 26s) (logmsgbot)
curprev 01:2801:28, 23 January 2016‎ imported>Labslogbot‎ 56,445 bytes +6,479‎ bd808@tin rebuilt wikiversions.php and synchronized wikiversions files: Temporarily back to 1.27.0-wmf11; need to rebuild l10n cache (logmsgbot)

curprev 01:1301:13, 22 January 2016‎ imported>Labslogbot‎ 49,966 bytes +7,942‎ catrope@tin Started scap: Deploying OATHAuth and WikimediaMessages i18n changes (logmsgbot)

curprev 01:1501:15, 21 January 2016‎ imported>Labslogbot‎ 42,024 bytes +7,498‎ Restarted logstash on logstash1003 (bd808)

curprev 01:2701:27, 20 January 2016‎ imported>Labslogbot‎ 34,526 bytes +5,797‎ aaron@tin Synchronized wmf-config: Configure $wgCdnReboundPurgeDelay (duration: 00m 32s) (logmsgbot)

curprev 23:2623:26, 18 January 2016‎ imported>Labslogbot‎ 28,729 bytes +1,261‎ krenair@tin Synchronized multiversion/MWMultiVersion.php: https://gerrit.wikimedia.org/r/264895 (duration: 00m 31s) (logmsgbot)

curprev 04:5804:58, 17 January 2016‎ imported>Labslogbot‎ 27,468 bytes +449‎ started restbase on restbase1002 (YuviPanda)

curprev 19:5219:52, 16 January 2016‎ imported>Labslogbot‎ 27,019 bytes +730‎ renaming and reimaging labcontrol2001 -> labtestweb2001 (andrewbogott)

curprev 22:4322:43, 15 January 2016‎ imported>Labslogbot‎ 26,289 bytes −484,770‎ aaron@tin Synchronized wmf-config/CommonSettings.php: Set $wgCentralAuthUseSlaves for testwiki (duration: 00m 33s) (logmsgbot)
curprev 01:0001:00, 15 January 2016‎ imported>Labslogbot‎ 511,059 bytes +2,038‎ krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryWatchlist.php: https://gerrit.wikimedia.org/r/#/c/264224/ (duration: 00m 31s) (logmsgbot)

curprev 01:2801:28, 14 January 2016‎ imported>Labslogbot‎ 509,021 bytes +4,993‎ krenair@tin rebuilt wikiversions.php and synchronized wikiversions files: (no message) (logmsgbot)

curprev 01:1901:19, 13 January 2016‎ imported>Labslogbot‎ 504,028 bytes +5,234‎ krenair@tin Synchronized php-1.27.0-wmf.10/extensions/Echo/Resources.php: https://gerrit.wikimedia.org/r/#/c/263645/ (duration: 00m 32s) (logmsgbot)

curprev 00:4600:46, 12 January 2016‎ imported>Labslogbot‎ 498,794 bytes +5,350‎ krenair@tin Synchronized wmf-config/InitialiseSettings.php: rv 443026e3ad18934dd0017a258673d88104cf6b5e (duration: 00m 29s) (logmsgbot)

curprev 01:1101:11, 11 January 2016‎ imported>Labslogbot‎ 493,444 bytes +1,147‎ deactivating eqiad<->GTT BGP peering, reported network issues (P2469) (paravoid)

curprev 18:2318:23, 9 January 2016‎ imported>Tim Landscheidt‎ m 492,297 bytes +228‎ Reverted edits by Tegel (talk) to last revision by Labslogbot

curprev 23:4923:49, 8 January 2016‎ imported>Labslogbot‎ 492,069 bytes +302‎ stalled puppet on carbon for now, messing with partman files (RobH)

curprev 23:2423:24, 7 January 2016‎ imported>Labslogbot‎ 491,767 bytes +1,399‎ repooled scb1002 for mobileapps (akosiaris)

curprev 23:0323:03, 6 January 2016‎ imported>Labslogbot‎ 490,368 bytes +361‎ switched restbase1009 to node 4.2 for testing, and restarted restbase; see https://phabricator.wikimedia.org/T107762 (gwicke)

curprev 22:3822:38, 5 January 2016‎ imported>Labslogbot‎ 490,007 bytes +921‎ aaron@tin Synchronized rpc: 830e1ed8d80295710dc02f18102b4fadae7fca86 (duration: 00m 55s) (logmsgbot)

curprev 20:5020:50, 4 January 2016‎ imported>Labslogbot‎ 489,086 bytes +1,072‎ ms-be1011 - powercycled, was frozen (mutante)

curprev 02:3202:32, 3 January 2016‎ imported>Labslogbot‎ 488,014 bytes +228‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Jan 3 02:31:58 UTC 2016 (duration 6m 52s) (logmsgbot)

curprev 03:3403:34, 2 January 2016‎ imported>Labslogbot‎ 487,786 bytes +313‎ deploying https://gerrit.wikimedia.org/r/261725, restarted apache2 on iridium (twentyafterfour)
curprev 01:0401:04, 2 January 2016‎ imported>Labslogbot‎ 487,473 bytes +602‎ imported vagrant 1.8.1 for jessie per bd808 (YuviPanda)

curprev 02:3102:31, 31 December 2015‎ imported>Labslogbot‎ 486,871 bytes +228‎ l10nupdate@tin ResourceLoader cache refresh completed at Thu Dec 31 02:31:43 UTC 2015 (duration 6m 52s) (logmsgbot)

curprev 15:0515:05, 30 December 2015‎ imported>Labslogbot‎ 486,643 bytes +973‎ restarting puppet on seaborgium; openldap will restart but config should not change (andrewbogott)
curprev 00:0800:08, 30 December 2015‎ imported>Labslogbot‎ 485,670 bytes +1,863‎ restarting nova-compute on labvirt1002 (andrewbogott)

curprev 01:0201:02, 29 December 2015‎ imported>Labslogbot‎ 483,807 bytes +2,394‎ awight@tin Synchronized php-1.27.0-wmf.9/extensions/CentralNotice: Update CentralNotice: T122251 (duration: 00m 34s) (logmsgbot)

curprev 03:0503:05, 27 December 2015‎ imported>Labslogbot‎ 481,413 bytes +482‎ run nodetool clearsnapshot -- v3 and nodetool clearsnapshot -- v1 on maps-test2001 (YuviPanda)

curprev 19:1219:12, 26 December 2015‎ imported>Labslogbot‎ 480,931 bytes +767‎ restarting varnish-frontend on cp3042 (paravoid)

curprev 15:5015:50, 25 December 2015‎ imported>Labslogbot‎ 480,164 bytes +449‎ testing new mariadb packages on db2070 (jynus)

curprev 23:2823:28, 24 December 2015‎ imported>Labslogbot‎ 479,715 bytes +1,399‎ powercycled mw1114 (mutante)
curprev 01:0501:05, 24 December 2015‎ imported>Labslogbot‎ 478,316 bytes +2,128‎ update payments from bae4d02afd8cfe1f8b8617c2f74bb36e420d281d to a7785baa7b40b442ecf0b60d47572502d0759780 (awight)

curprev 00:4000:40, 23 December 2015‎ imported>Labslogbot‎ 476,188 bytes +3,023‎ krenair@tin Synchronized wmf-config/CommonSettings-labs.php: https://gerrit.wikimedia.org/r/260696 & https://gerrit.wikimedia.org/r/260699 (duration: 05m 28s) (logmsgbot)

curprev 00:2900:29, 22 December 2015‎ imported>Labslogbot‎ 473,165 bytes +3,038‎ krenair@tin Synchronized php-1.27.0-wmf.9/extensions/VisualEditor: https://gerrit.wikimedia.org/r/#/c/260492/ (duration: 00m 32s) (logmsgbot)

curprev 23:2423:24, 20 December 2015‎ imported>Labslogbot‎ 470,127 bytes +878‎ Katie and Jeff paged about bellatrix (Reedy)

curprev 21:5521:55, 19 December 2015‎ imported>Labslogbot‎ 469,249 bytes +1,169‎ restarted zotero on sca1001, various OOM messages (_joe_)
curprev 01:0201:02, 19 December 2015‎ imported>Labslogbot‎ 468,080 bytes +3,087‎ entire restbase cluster: removed 5% root reserve from data partition with tune2fs -m 0 /dev/mapper/restbase$NODE--vg-{srv,var} (gwicke)

curprev 01:2501:25, 18 December 2015‎ imported>Labslogbot‎ 464,993 bytes +6,803‎ in preparation for Iaefb2d191e, disabling puppet on mc* and rdb* (ori)

curprev 00:2800:28, 17 December 2015‎ imported>Labslogbot‎ 458,190 bytes +5,451‎ ori@tin Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: I552cf6b0420: Upgrade some ApiStashEdit logging calls to info() (duration: 00m 30s) (logmsgbot)

curprev 00:4000:40, 16 December 2015‎ imported>Labslogbot‎ 452,739 bytes +7,890‎ catrope@tin Synchronized wmf-config/InitialiseSettings.php: Enable wmgGraphEnableGzip on all wikis (duration: 00m 30s) (logmsgbot)

curprev 00:3400:34, 15 December 2015‎ imported>Labslogbot‎ 444,849 bytes +4,803‎ puppet disabled on silver since last Puppet run was at Fri Dec 11 15:27:28 UTC 2015 due to 'testing osm debugging' (Reedy)

curprev 11:1011:10, 13 December 2015‎ imported>Labslogbot‎ 440,046 bytes +682‎ elastic in eqiad: restarting elastic1012 to release opened log filedesc (dcausse)

curprev 22:1722:17, 12 December 2015‎ imported>Labslogbot‎ 439,364 bytes +1,510‎ updated fundraising dashboard from 961a24c9cf76966aaa7ba8c60e13b6d1d37fa859 to 59e51c4ff74c3c584daf6c5de3bb66daa764cd28 (ejegg)
curprev 00:3000:30, 12 December 2015‎ imported>Labslogbot‎ 437,854 bytes +3,848‎ restarting slapd on seaborgium with manual hack (paravoid)

curprev 01:1301:13, 11 December 2015‎ imported>Labslogbot‎ 434,006 bytes +6,887‎ ori@tin Synchronized php-1.27.0-wmf.8/includes/jobqueue: Ia44ec5ed4: Add per-partition JobQueueRedis aggregation + Fix bad regex in 6fe2f48df (duration: 00m 31s) (logmsgbot)

curprev 01:2501:25, 10 December 2015‎ imported>Labslogbot‎ 427,119 bytes +7,091‎ ebernhardson@tin Synchronized php-1.27.0-wmf.7/extensions/CirrusSearch/: Add master timeout parameter to mapping updates so it can be increased when the master is slow (duration: 00m 30s) (logmsgbot)

curprev 01:1901:19, 9 December 2015‎ imported>Labslogbot‎ 420,028 bytes +5,085‎ Updating paymentswiki from 96d8a49ecd521ac8ce75521aaf97ed8f0dbefc9a to 74143e43eb36f93be7881f626443182d3bb58cef (awight)

curprev 01:1601:16, 8 December 2015‎ imported>Labslogbot‎ 414,943 bytes +6,034‎ catrope@tin Synchronized wmf-config/InitialiseSettings.php: Also blacklist Minerva for WPB (duration: 00m 28s) (logmsgbot)

curprev 21:4821:48, 6 December 2015‎ imported>Labslogbot‎ 408,909 bytes +515‎ krypton unresponsive, nothing on console. shutting down, increasing instance ram from 2 to 4g, and rebooting. (ori)

curprev 18:3018:30, 5 December 2015‎ imported>Labslogbot‎ 408,394 bytes +821‎ started nodetool decommission on restbase1008 (gwicke)

curprev 22:2022:20, 4 December 2015‎ imported>Labslogbot‎ 407,573 bytes +1,808‎ updated dash from bed9895563fc0f5d3b96791fae6ed104b7778f0d to 9f0a4d98586f1281716e39b34d924de524458177 (ejegg)
curprev 00:3500:35, 4 December 2015‎ imported>Labslogbot‎ 405,765 bytes +4,063‎ catrope@tin Synchronized php-1.27.0-wmf.7/extensions/CentralNotice: SWAT (duration: 00m 32s) (logmsgbot)

curprev 00:1800:18, 3 December 2015‎ imported>Labslogbot‎ 401,702 bytes +2,268‎ Deploy paymentswiki config for T112181 (awight)

curprev 00:2900:29, 2 December 2015‎ imported>Labslogbot‎ 399,434 bytes +1,871‎ puppetstoredconfigclean.rb labcontrol2001.wikimedia.org fixes icinga config (mutante)

curprev 00:5100:51, 1 December 2015‎ imported>Labslogbot‎ 397,563 bytes +3,905‎ updated fundraising dash from bc0ec4bb2427ebeb2d4cd60eda65e3d82d006937 to 2ee15180150c8b685b14abdbf7c6769736fb49a0 (ejegg)

curprev 01:0501:05, 30 November 2015‎ imported>Labslogbot‎ 393,658 bytes +1,159‎ bd808@tin sync-l10n completed (1.27.0-wmf.7) (duration: 01m 19s) (logmsgbot)

curprev 22:4822:48, 28 November 2015‎ imported>Labslogbot‎ 392,499 bytes +657‎ bd808@tin Synchronized php-1.27.0-wmf.5/cache/l10n: bd808 testing l10nupdate sync-dir using stale branch (duration: 01m 29s) (logmsgbot)

curprev 21:2321:23, 27 November 2015‎ imported>Labslogbot‎ 391,842 bytes +2,480‎ krenair@tin Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/255726/1 (duration: 00m 31s) (logmsgbot)

curprev 20:5920:59, 26 November 2015‎ imported>Labslogbot‎ 389,362 bytes +2,051‎ aaron@tin Synchronized wmf-config/jobqueue-eqiad.php: Adjust connection timeout and maxPartitionsTry (duration: 00m 34s) (logmsgbot)
curprev 01:0301:03, 26 November 2015‎ imported>Labslogbot‎ 387,311 bytes +3,058‎ from f867beecd3a4fb596e2d08c8d2b0e4173ad0b356 to d5afa10383cfd41adbe798c3ed2aa97839be8bc2 (awight)

curprev 01:0301:03, 25 November 2015‎ imported>Labslogbot‎ 384,253 bytes +4,146‎ re-armed keyholders on mira (mutante)

curprev 01:0001:00, 24 November 2015‎ imported>Labslogbot‎ 380,107 bytes +3,523‎ updated payments-wiki from ecbc49c63b259a4363c51491ddff273e3579f0e9 to 5358960d6857f589939c9b86ea37af168a4f8b0b (ejegg)

curprev 01:2701:27, 23 November 2015‎ imported>Labslogbot‎ 376,584 bytes +291‎ end enwiki test election from cli to avoid confusion with real election (Jamesofur)

curprev 19:1419:14, 21 November 2015‎ imported>Labslogbot‎ 376,293 bytes +603‎ updated fundraising tools from d7253eb1c6c9478ce2cf8760279d2ad1b7f0a132 to ebed29c0eccf38c812b20a957b3487a15bfa9cbc (ejegg|away)
curprev 00:5000:50, 21 November 2015‎ imported>Labslogbot‎ 375,690 bytes +2,605‎ update-ubuntu-mirror on carbon completed successfully ; going to re-enable icinga-wm (ori)

curprev 01:2901:29, 20 November 2015‎ imported>Labslogbot‎ 373,085 bytes +3,783‎ catrope@tin Synchronized php-1.27.0-wmf.7/extensions/Echo/includes/formatters/BasicFormatter.php: unstub wgLang to fix type hint errors (duration: 00m 20s) (logmsgbot)

curprev 00:2900:29, 19 November 2015‎ imported>Labslogbot‎ 369,302 bytes +4,880‎ catrope@tin Synchronized php-1.27.0-wmf.7/extensions/Flow/: SWAT: better permissions errors for opt-in (duration: 00m 21s) (logmsgbot)

curprev 01:2201:22, 18 November 2015‎ imported>Labslogbot‎ 364,422 bytes +4,679‎ rebooting mw1158 (andrewbogott)

curprev 00:2100:21, 17 November 2015‎ imported>Labslogbot‎ 359,743 bytes +3,042‎ krinkle@tin Synchronized wmf-config/jobqueue-codfw.php: (no message) (duration: 00m 26s) (logmsgbot)

curprev 18:0318:03, 15 November 2015‎ imported>Labslogbot‎ 356,701 bytes +1,175‎ disabling puppet, restarting mysql on es2008, es2009, es2010 for configuration check/upgrade (jynus)

curprev 23:3123:31, 14 November 2015‎ imported>Labslogbot‎ 355,526 bytes +1,784‎ dsh delete php-1.26wmf22 23 and 24 (Reedy)

curprev 23:3123:31, 13 November 2015‎ imported>Labslogbot‎ 353,742 bytes +2,055‎ ori@tin Synchronized rpc/RunJobs.php: I5e15ec9fb: Disable ChronologyProtector for rpc/RunJobs.php (duration: 00m 30s) (logmsgbot)
curprev 01:2701:27, 13 November 2015‎ imported>Labslogbot‎ 351,687 bytes +4,168‎ bounced mw1045 HHVM for ebernhardson (YuviPanda)

curprev 01:0201:02, 12 November 2015‎ imported>Labslogbot‎ 347,519 bytes +4,838‎ restbase: finished full deploy of restbase-deploy ce4abe784 (gwicke)

curprev 00:5800:58, 11 November 2015‎ imported>Labslogbot‎ 342,681 bytes +5,075‎ added jenkins_1.625.1 to APT repo. precise-wikimedia (mutante)

curprev 00:1300:13, 10 November 2015‎ imported>Labslogbot‎ 337,606 bytes +3,724‎ Attached Richardelainechambers@commonswiki to the global account of the same name (hoo)

curprev 20:4320:43, 8 November 2015‎ imported>Labslogbot‎ 333,882 bytes +854‎ krenair@tin Synchronized php-1.27.0-wmf.5/extensions/ConfirmEdit/SimpleCaptcha/Captcha.php: https://gerrit.wikimedia.org/r/#/c/251673/ (duration: 00m 34s) (logmsgbot)

curprev 19:5019:50, 7 November 2015‎ imported>Labslogbot‎ 333,028 bytes +1,574‎ krenair@tin Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: https://gerrit.wikimedia.org/r/#/c/251739/ (duration: 00m 35s) (logmsgbot)
curprev 00:2300:23, 7 November 2015‎ imported>Labslogbot‎ 331,454 bytes +3,429‎ maxsem@tin Synchronized php-1.27.0-wmf.5/extensions/WikimediaMaintenance/getPageCounts.php: (no message) (duration: 00m 34s) (logmsgbot)

curprev 01:2801:28, 6 November 2015‎ imported>Labslogbot‎ 328,025 bytes +4,445‎ bd808@tin Synchronized wmf-config/InitialiseSettings.php: Touch (duration: 00m 35s) (logmsgbot)

curprev 00:1300:13, 5 November 2015‎ imported>Labslogbot‎ 323,580 bytes +5,250‎ updated fundraising tools from e1b60fa2c258fd4ff55905b03a4d8886132278c1 to f2f571903d32ec73aeab73ce58c6bee71717213b (ejegg)

curprev 01:1001:10, 4 November 2015‎ imported>Labslogbot‎ 318,330 bytes +4,098‎ krenair@tin Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: https://gerrit.wikimedia.org/r/#/c/250874/ (duration: 00m 17s) (logmsgbot)

curprev 01:0501:05, 3 November 2015‎ imported>Labslogbot‎ 314,232 bytes +6,334‎ enabled CRM module amazon_audit (awight)

curprev 16:5916:59, 1 November 2015‎ imported>Labslogbot‎ 307,898 bytes +444‎ restarting cassandra on aqs1002; was out of heap space (gwicke)

curprev 08:5708:57, 31 October 2015‎ imported>Labslogbot‎ 307,454 bytes +626‎ elastic1008 deleting /var/log/elasticsearch/production-search-eqiad_index_indexing_slowlog.log.[2-7] (dcausse)

curprev 23:5323:53, 30 October 2015‎ imported>Labslogbot‎ 306,828 bytes +4,432‎ ori@tin Synchronized wmf-config/CommonSettings.php: Ifd3543d99: Re-enabled sidebar cache per 47eb083a0fe4 (duration: 00m 17s) (logmsgbot)
curprev 00:4800:48, 30 October 2015‎ imported>Labslogbot‎ 302,396 bytes +8,055‎ tgr@tin Synchronized php-1.27.0-wmf.4/includes/specials/SpecialUserlogin.php: T117027 (duration: 00m 18s) (logmsgbot)

curprev 01:0601:06, 29 October 2015‎ imported>Labslogbot‎ 294,341 bytes +5,787‎ lowered gc_grace on wikipedia parsoid html and data-parsoid keyspaces to 24 hours (gwicke)

curprev 00:5500:55, 28 October 2015‎ imported>Labslogbot‎ 288,554 bytes +5,206‎ deployed graphoid - https://gerrit.wikimedia.org/r/#/c/249324/ (yurik)

curprev 01:0201:02, 27 October 2015‎ imported>Labslogbot‎ 283,348 bytes +4,898‎ krinkle@tin Synchronized php-1.27.0-wmf.3/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: T116693 (duration: 00m 19s) (logmsgbot)

curprev 05:3605:36, 25 October 2015‎ imported>Labslogbot‎ 278,450 bytes +373‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Oct 25 05:36:33 UTC 2015 (duration 36m 32s) (logmsgbot)

curprev 19:3419:34, 24 October 2015‎ imported>Labslogbot‎ 278,077 bytes +707‎ deployed https://gerrit.wikimedia.org/r/#/c/248638/ and restarted apache on iridium (twentyafterfour)
curprev 00:2100:21, 24 October 2015‎ imported>Labslogbot‎ 277,370 bytes +3,609‎ started second machine (nobelium) performing copy of elasticsearch indices to codfw with 40 threads (ebernhardson)

curprev 00:1000:10, 23 October 2015‎ imported>Labslogbot‎ 273,761 bytes +5,419‎ krenair@tin Synchronized w/static/images/project-logos/anwiki.png: https://gerrit.wikimedia.org/r/#/c/247253/ (duration: 00m 16s) (logmsgbot)

curprev 01:1001:10, 22 October 2015‎ imported>Labslogbot‎ 268,342 bytes +6,010‎ krenair@mira Synchronized README: testing sync from mira (duration: 00m 17s) (logmsgbot)

curprev 01:1201:12, 21 October 2015‎ imported>Labslogbot‎ 262,332 bytes +7,026‎ hoo@tin Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 17s) (logmsgbot)

curprev 23:4323:43, 19 October 2015‎ imported>Labslogbot‎ 255,306 bytes +4,671‎ ebernhardson@tin Synchronized wmf-config/InitialiseSettings.php: testwiki Graphoid to restbase (duration: 00m 17s) (logmsgbot)

curprev 10:0410:04, 18 October 2015‎ imported>Labslogbot‎ 250,635 bytes +994‎ uploaded to apt.wikimedia.org trusty-wikimedia: apertium-mlt-ara_0.1.0~r57554-1 (akosiaris)

curprev 20:0220:02, 17 October 2015‎ imported>Labslogbot‎ 249,641 bytes +751‎ powercycle analytics1034, no console no ssh (godog)

curprev 20:5220:52, 16 October 2015‎ imported>Labslogbot‎ 248,890 bytes +1,454‎ Restarting Jenkins to remove potential dead locks before the week-end (hashar)

curprev 22:0922:09, 15 October 2015‎ imported>Labslogbot‎ 247,436 bytes +1,474‎ don't worry, _joe_ was around and we approved Roan's last deploy as an exception :) (greg-g)

curprev 23:4623:46, 14 October 2015‎ imported>Labslogbot‎ 245,962 bytes +3,718‎ demon@tin Synchronized wmf-config/CommonSettings.php: Nevermind on skipping timed text, extension config is weird (duration: 00m 17s) (logmsgbot)
curprev 00:1900:19, 14 October 2015‎ imported>Labslogbot‎ 242,244 bytes +4,657‎ updated civicrm from c7af7634e75eb8702f5e16081a86ab86ce69c7c2 to ba39f3181431c8409416e580dcf6e15ac5f96a21 (ejegg)

curprev 00:5100:51, 13 October 2015‎ imported>Labslogbot‎ 237,587 bytes +4,478‎ catrope@tin Synchronized wmf-config/InitialiseSettings.php: Reapply: Flow-occupy talk namespaces on sewikimedia (duration: 01m 13s) (logmsgbot)

curprev 04:5704:57, 11 October 2015‎ imported>Labslogbot‎ 233,109 bytes +373‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Oct 11 04:57:40 UTC 2015 (duration 57m 39s) (logmsgbot)

curprev 14:5114:51, 10 October 2015‎ imported>Southparkfan‎ 232,736 bytes −666,193‎ Archive SAL

curprev 22:2022:20, 9 October 2015‎ imported>Labslogbot‎ 898,929 bytes +1,949‎ took down Fundraising campaigns (ejegg)
curprev 01:2401:24, 9 October 2015‎ imported>Labslogbot‎ 896,980 bytes +3,359‎ catrope@tin Synchronized php-1.27.0-wmf.2/extensions/CentralNotice/resources/subscribing/ext.centralNotice.display.js: Add period to try to flush out ResourceLoader issue (duration: 01m 26s) (logmsgbot)

curprev 01:1701:17, 8 October 2015‎ imported>Labslogbot‎ 893,621 bytes +5,436‎ phabricator is now up and running again (twentyafterfour)

curprev 00:0500:05, 7 October 2015‎ imported>Labslogbot‎ 888,185 bytes +2,731‎ catrope@tin Synchronized php-1.27.0-wmf.2/extensions/Echo: SWAT (duration: 00m 17s) (logmsgbot)

curprev 23:5323:53, 5 October 2015‎ imported>Labslogbot‎ 885,454 bytes +2,642‎ krenair@tin Synchronized php-1.27.0-wmf.1/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/243833/ (duration: 00m 17s) (logmsgbot)

curprev 23:3623:36, 4 October 2015‎ imported>Labslogbot‎ 882,812 bytes +580‎ ori@tin Synchronized php-1.27.0-wmf.1/extensions/ContentTranslation/extension.json: 8c80ec1273: Updated mediawiki/core Project: mediawiki/extensions/ContentTranslation (duration: 00m 17s) (logmsgbot)

curprev 15:3315:33, 3 October 2015‎ imported>Labslogbot‎ 882,232 bytes +706‎ stopping temporarily labsdb1004 mariadb to complete clone process (jynus)

curprev 22:4622:46, 2 October 2015‎ imported>Labslogbot‎ 881,526 bytes +2,456‎ holmium: apt-get clean for a little disk space - /var/log/designate/designate-mdns.log is more than half the size of / - needs logrotate (mutante)
curprev 00:4400:44, 2 October 2015‎ imported>Labslogbot‎ 879,070 bytes +5,530‎ ori@tin Synchronized php-1.27.0-wmf.1/includes/resourceloader: I21bb3f08e7f and follow-ups (duration: 00m 18s) (logmsgbot)

curprev 22:5522:55, 30 September 2015‎ imported>Labslogbot‎ 873,540 bytes +2,561‎ twentyafterfour@tin Synchronized php-1.27.0-wmf.1/: deploying several fixes to the branch (duration: 01m 52s) (logmsgbot)
curprev 01:0201:02, 30 September 2015‎ imported>Labslogbot‎ 870,979 bytes +4,769‎ ori@tin Synchronized php-1.26wmf24/vendor: 940124a7db: Updated mediawiki/core Project: mediawiki/vendor ff5e254f7eddf811f6f66b4a4063b1a8cc70f265 (duration: 00m 21s) (logmsgbot)

curprev 01:1301:13, 29 September 2015‎ imported>Labslogbot‎ 866,210 bytes +2,479‎ didnt powercycle analytics1035 yet, it recovered on its own. (robh)

curprev 23:1723:17, 27 September 2015‎ imported>Labslogbot‎ 863,731 bytes +851‎ ori@tin Synchronized php-1.26wmf24/./resources/src/mediawiki.toolbar/toolbar.less: I94ced06178: mediawiki.toolbar: temporary workaround for T113868 (duration: 00m 17s) (logmsgbot)

curprev 19:0419:04, 26 September 2015‎ imported>Labslogbot‎ 862,880 bytes +189‎ restarting Jenkins. Just in case :-D (hashar)

curprev 23:5423:54, 25 September 2015‎ imported>Labslogbot‎ 862,691 bytes +2,772‎ ori@tin Synchronized php-1.26wmf24/includes/GlobalFunctions.php: 37c6972f94: Made wfIsBadImage() use APC (duration: 00m 17s) (logmsgbot)
curprev 01:2801:28, 25 September 2015‎ imported>Labslogbot‎ 859,919 bytes +5,372‎ krinkle@tin Synchronized php-1.26wmf24/extensions/WikimediaEvents: I5608f8ffd1c - Fix trailing comma (duration: 00m 17s) (logmsgbot)

curprev 01:1501:15, 24 September 2015‎ imported>Labslogbot‎ 854,547 bytes +4,662‎ ori@tin Synchronized php-1.26wmf24/includes: Ifa0d4cfe8e3: Backport I1ff61153d and I8e4c3d5a5 (duration: 00m 23s) (logmsgbot)

curprev 00:5600:56, 23 September 2015‎ imported>Labslogbot‎ 849,885 bytes +4,192‎ legoktm@tin Synchronized php-1.26wmf24/extensions/Echo/Hooks.php: Remove duplicate 'MediaWiki' prefix from echo.unseen stats (duration: 00m 12s) (logmsgbot)

curprev 00:3200:32, 22 September 2015‎ imported>Labslogbot‎ 845,693 bytes +4,741‎ Disabled Puppet for 24h on hafnium and stopped ganglia-monitor. gmond was saturating CPU. (ori)

curprev 22:3422:34, 20 September 2015‎ imported>Labslogbot‎ 840,952 bytes +735‎ reloda pybal on lvs1012 (yuvipanda)

curprev 23:1223:12, 19 September 2015‎ imported>Labslogbot‎ 840,217 bytes +1,606‎ begining Cassandra repair on restbase1005 (nodetool repair -pr) (urandom)

curprev 23:2223:22, 18 September 2015‎ imported>Labslogbot‎ 838,611 bytes +4,102‎ update fundraising-tools from 3e0e3ae799a507b378d0ece3e71631b10b361329 to e1b60fa2c258fd4ff55905b03a4d8886132278c1 (awight)
curprev 00:1400:14, 18 September 2015‎ imported>Labslogbot‎ 834,509 bytes +6,909‎ restarted logstash on logstash1001 (ori)

curprev 00:3500:35, 17 September 2015‎ imported>Labslogbot‎ 827,600 bytes +6,029‎ updated payments from 155cdeb737c01baf62551292764fd2f5a93a9a63 to 1bdd287b083032ff418434ad6bb6920735af918a (cwdent)

curprev 01:0301:03, 16 September 2015‎ imported>Labslogbot‎ 821,571 bytes +6,779‎ krinkle@tin Synchronized php-1.26wmf23/resources/src/mediawiki/mediawiki.js: hotfix Ia2fcd13f4 (duration: 00m 12s) (logmsgbot)

curprev 00:5000:50, 15 September 2015‎ imported>Labslogbot‎ 814,792 bytes +4,143‎ reinstalling lvs300[34] to jessie (bblack)

curprev 06:0206:02, 13 September 2015‎ imported>Labslogbot‎ 810,649 bytes +363‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 13 06:02:52 UTC 2015 (duration 2m 51s) (logmsgbot)

curprev 20:1520:15, 12 September 2015‎ imported>Labslogbot‎ 810,286 bytes +677‎ Rolling back Echo to 1.26wmf21 branch on mw1017 (testwiki) to measure increase in render-blocking CSS size (ori)

curprev 21:2121:21, 11 September 2015‎ imported>Labslogbot‎ 809,609 bytes +1,756‎ shutdown nodepool on labnodepool1001.eqiad.wmnet until monday (hashar)
curprev 01:1601:16, 11 September 2015‎ imported>Labslogbot‎ 807,853 bytes +4,964‎ ori@tin Synchronized php-1.26wmf22/extensions/TitleBlacklist: 9bf13dbe0b, 3203b045f7 (duration: 00m 12s) (logmsgbot)

curprev 01:0401:04, 10 September 2015‎ imported>Labslogbot‎ 802,889 bytes +5,413‎ ori@tin Synchronized php-1.26wmf21/extensions/NavigationTiming: I2605c746b: Ensure timings are reported after the page has loaded (duration: 00m 13s) (logmsgbot)

curprev 00:0000:00, 9 September 2015‎ imported>Labslogbot‎ 797,476 bytes +4,248‎ catrope@tin Finished scap: Need to update i18n for a new Echo message (duration: 23m 08s) (logmsgbot)

curprev 00:4600:46, 8 September 2015‎ imported>Labslogbot‎ 793,228 bytes +1,579‎ mwscript deleteEqualMessages.php --wiki eswiki (Krinkle)

curprev 04:2804:28, 6 September 2015‎ imported>Labslogbot‎ 791,649 bytes +364‎ l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 6 04:27:57 UTC 2015 (duration 27m 56s) (logmsgbot)

curprev 23:3723:37, 5 September 2015‎ imported>Labslogbot‎ 791,285 bytes +434‎ mwscript deleteEqualMessages.php --wiki fywiktionary (Krinkle)

curprev 23:5223:52, 4 September 2015‎ imported>Labslogbot‎ 790,851 bytes +2,618‎ mattflaschen@tin Synchronized wmf-config/InitialiseSettings-labs.php: Beta-only change (duration: 00m 12s) (logmsgbot)
curprev 00:1100:11, 4 September 2015‎ imported>Labslogbot‎ 788,233 bytes +6,425‎ krinkle@tin Synchronized php-1.26wmf21/includes/resourceloader/ResourceLoader.php: I24f68e34a9fa4918 (duration: 00m 12s) (logmsgbot)

curprev 00:3600:36, 3 September 2015‎ imported>Labslogbot‎ 781,808 bytes +5,582‎ ori@tin Synchronized php-1.26wmf21/includes/parser/Preprocessor_Hash.php: Idd1acd903: Decline to cache preprocessor items larger than 1 Mb (duration: 00m 11s) (logmsgbot)

curprev 00:3300:33, 2 September 2015‎ imported>Labslogbot‎ 776,226 bytes +4,769‎ krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/235366/ (duration: 00m 13s) (logmsgbot)

curprev 23:5623:56, 31 August 2015‎ imported>Labslogbot‎ 771,457 bytes +4,638‎ krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/233665/ (duration: 00m 11s) (logmsgbot)

curprev 12:5812:58, 30 August 2015‎ imported>Labslogbot‎ 766,819 bytes +1,020‎ lvchange -ay labstore/others on labstore1002 (godog)

curprev 15:2615:26, 29 August 2015‎ imported>Labslogbot‎ 765,799 bytes +863‎ killing idle mysql connections from phabricator and setting wait and interactive timeout to 60 (jynus)

curprev 23:4523:45, 28 August 2015‎ imported>Labslogbot‎ 764,936 bytes +2,516‎ krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/234679/ (duration: 06m 56s) (logmsgbot)
curprev 00:2400:24, 28 August 2015‎ imported>Labslogbot‎ 762,420 bytes +5,788‎ powercycled mw2027 (mutante)

curprev 00:3100:31, 27 August 2015‎ imported>Labslogbot‎ 756,632 bytes +6,046‎ finished phabricator upgrade, everything appears to be working (twentyafterfour)

curprev 00:2800:28, 26 August 2015‎ imported>Labslogbot‎ 750,586 bytes +5,150‎ ori@tin Synchronized php-1.26wmf20/extensions/Scribunto/engines/LuaCommon/LuaCommon.php: (no message) (duration: 00m 13s) (logmsgbot)

curprev 23:4623:46, 24 August 2015‎ imported>Labslogbot‎ 745,436 bytes +3,386‎ mattflaschen@tin Synchronized wmf-config: Remove wgFlowOccupyPages (duration: 00m 12s) (logmsgbot)

curprev 16:5416:54, 23 August 2015‎ imported>Labslogbot‎ 742,050 bytes +219‎ bouncing Cassandra on restbase1001 to apply temporary GC settings (urandom)

curprev 23:0823:08, 22 August 2015‎ imported>Labslogbot‎ 741,831 bytes +953‎ krenair@tin Synchronized php-1.26wmf19/extensions/AbuseFilter/maintenance/addMissingLoggingEntries.php: (no message) (duration: 01m 05s) (logmsgbot)
curprev 00:2600:26, 22 August 2015‎ imported>Labslogbot‎ 740,878 bytes +1,894‎ deleting blog.sh and blog_pageviews crontab from stat1003 (mutante)

curprev 23:4023:40, 20 August 2015‎ imported>Labslogbot‎ 738,984 bytes +4,846‎ ebernhardson@tin Synchronized php-1.26wmf19/extensions/CirrusSearch/: Fix some cirrussearch logspam (duration: 00m 13s) (logmsgbot)

curprev 21:2021:20, 19 August 2015‎ imported>Labslogbot‎ 734,138 bytes +4,027‎ livehack reverted sodium back to normal, testing done (robh)
curprev 01:1101:11, 19 August 2015‎ imported>Labslogbot‎ 730,111 bytes +3,199‎ updated kartotherian (yurik)

curprev 00:0200:02, 18 August 2015‎ imported>Labslogbot‎ 726,912 bytes +3,254‎ analytics1041 down, attempting power cycle (ottomata)

curprev 18:1518:15, 16 August 2015‎ imported>Labslogbot‎ 723,658 bytes +694‎ krenair@tin Synchronized wmf-config/extension-list: https://gerrit.wikimedia.org/r/#/c/169612/ - remove Extension:Oversight (duration: 00m 21s) (logmsgbot)

curprev 22:5622:56, 15 August 2015‎ imported>Labslogbot‎ 722,964 bytes +961‎ removed 13 bounce_records for User:odder from bouncehandler database (legoktm)
curprev 01:1501:15, 15 August 2015‎ imported>Labslogbot‎ 722,003 bytes +2,802‎ starting broker on analytics1012, camus wasn't happy about that either. hrm. (ottomata)

curprev 23:4523:45, 13 August 2015‎ imported>Labslogbot‎ 719,201 bytes +6,324‎ rollback paymentswiki from 2e7b449224317779d53ff84527166c0d378a0a40 to 823393264d6795bbaec490ff86f17580f722e598 (awight)
curprev 01:2601:26, 13 August 2015‎ imported>Labslogbot‎ 712,877 bytes +3,637‎ Restarted conversion of support desk from LQT->Flow using convertLqtPageOnLocalWiki.php, using hhvm on mw1041 (matt_flaschen)

curprev 01:2201:22, 12 August 2015‎ imported>Labslogbot‎ 709,240 bytes +5,841‎ zirconium - shut down, i'm sure, mollyguard (mutante)

curprev 01:2001:20, 11 August 2015‎ imported>Labslogbot‎ 703,399 bytes +4,246‎ ori@tin Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoader.php: I2089b21fc: Revert resourceloader: Add must-revalidate to Cache-Control (duration: 00m 12s) (logmsgbot)

curprev 17:5717:57, 9 August 2015‎ imported>Labslogbot‎ 699,153 bytes +783‎ issuing nodetool cleanup on restbase1006 (urandom)

curprev 14:5814:58, 8 August 2015‎ imported>Labslogbot‎ 698,370 bytes +1,003‎ issuing nodetool cleanup on restbase1005 (urandom)

curprev 21:0021:00, 7 August 2015‎ imported>Labslogbot‎ 697,367 bytes +2,551‎ krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/230227/ - should be a noop for prod (duration: 00m 12s) (logmsgbot)
curprev 01:0801:08, 7 August 2015‎ imported>Labslogbot‎ 694,816 bytes +7,694‎ ori Synchronized php-1.26wmf17/extensions/Gadgets: I2089b21fc: Updated mediawiki/core Project: mediawiki/extensions/Gadgets (duration: 00m 13s) (logmsgbot)

curprev 00:4900:49, 6 August 2015‎ imported>Labslogbot‎ 687,122 bytes +7,735‎ reset password for User:Tonval after identify verification (Jamesofur)

curprev 23:5923:59, 4 August 2015‎ imported>Labslogbot‎ 679,387 bytes +4,864‎ maxsem Synchronized php-1.26wmf16/extensions/WikimediaEvents/: SWAT (duration: 00m 12s) (logmsgbot)
curprev 00:0200:02, 4 August 2015‎ imported>Labslogbot‎ 674,523 bytes +6,039‎ updated paymentswiki to a8c0ecbedef6179c78ed833da9f2049cb0f2641b (awight)

curprev 00:0300:03, 3 August 2015‎ imported>Labslogbot‎ 668,484 bytes +789‎ krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/228198/ (duration: 00m 12s) (logmsgbot)

curprev 06:0506:05, 1 August 2015‎ imported>Labslogbot‎ 667,695 bytes +491‎ removing some old apache access logs from mw1114 (_joe_)
curprev 00:1200:12, 1 August 2015‎ imported>Labslogbot‎ 667,204 bytes +3,845‎ ori Synchronized extract2.php: Ie919881a4: Add an API listing template to the allowed templates in extract2.php (logmsgbot)

curprev 00:3500:35, 31 July 2015‎ imported>Labslogbot‎ 663,359 bytes +7,505‎ catrope Synchronized php-1.26wmf16/extensions/Flow/includes/Model/WikiReference.php: debugging (duration: 00m 12s) (logmsgbot)

curprev 00:4800:48, 30 July 2015‎ imported>Labslogbot‎ 655,854 bytes +7,841‎ ori Synchronized php-1.26wmf15/includes/Message.php: 160f69871c: Debug logging for T102199 (duration: 00m 13s) (logmsgbot)

curprev 00:4300:43, 29 July 2015‎ imported>Labslogbot‎ 648,013 bytes +4,914‎ ori Synchronized php-1.26wmf15/extensions/AbuseFilter: Revert "Revert "Conversion to using getMainStashInstance()"" (duration: 00m 12s) (logmsgbot)

curprev 01:1101:11, 28 July 2015‎ imported>Labslogbot‎ 643,099 bytes +5,843‎ krenair Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/227371/ (duration: 00m 11s) (logmsgbot)

curprev 01:1801:18, 27 July 2015‎ imported>Labslogbot‎ 637,256 bytes +1,228‎ Re-pooling mw1159 and mw1160; ran out of time for debugging. (ori)

curprev 20:5120:51, 25 July 2015‎ imported>Labslogbot‎ 636,028 bytes +1,141‎ rolling restart of restbase instances (gwicke)

curprev 21:5721:57, 24 July 2015‎ imported>Labslogbot‎ 634,887 bytes +2,829‎ running mwscript populateContentModel.php --wiki=enwiki --ns=all --table=page (legoktm)
curprev 00:2100:21, 24 July 2015‎ imported>Labslogbot‎ 632,058 bytes +5,446‎ Re-enabled Puppet on mw1153 (ori)

curprev 01:0501:05, 23 July 2015‎ imported>Labslogbot‎ 626,612 bytes +6,733‎ phab is back (twentyafterfour)

curprev 23:4523:45, 21 July 2015‎ imported>Labslogbot‎ 619,879 bytes +4,786‎ catrope Synchronized wmf-config/InitialiseSettings.php: Set $wgVectorResponsive = true on testwiki (duration: 00m 12s) (logmsgbot)

curprev 23:4423:44, 20 July 2015‎ imported>Labslogbot‎ 615,093 bytes +1,285‎ removed experimental nodes (1008, 1009) from system.peers on production C* nodes (gwicke)
curprev 00:0200:02, 20 July 2015‎ imported>Labslogbot‎ 613,808 bytes +1,946‎ DNS update - adding language "azb" to langlist (mutante)

curprev 20:5820:58, 18 July 2015‎ imported>Labslogbot‎ 611,862 bytes +821‎ legoktm Synchronized wmf-config/InitialiseSettings-labs.php: labs only (duration: 00m 12s) (logmsgbot)
curprev 00:4900:49, 18 July 2015‎ imported>Labslogbot‎ 611,041 bytes +2,129‎ restored recurring globalcollect batch size of 250 (ejegg)

curprev 21:2721:27, 16 July 2015‎ imported>Labslogbot‎ 608,912 bytes +3,677‎ bounced nutcracker on mw1139 as well. hashar noticed flood of errors from these hosts on https://logstash.wikimedia.org/#/dashboard/elasticsearch/mediawiki-errors . lack of monitoring / alerts is troubling. (ori)
curprev 01:2201:22, 16 July 2015‎ imported>Labslogbot‎ 605,235 bytes +7,251‎ es1.6 upgrade: upgrade elastic1023 (manybubbles)

curprev 23:4623:46, 14 July 2015‎ imported>Labslogbot‎ 597,984 bytes +3,095‎ es1.6 upgrade: upgraded elastic1011 (manybubbles)
curprev 01:2201:22, 14 July 2015‎ imported>Labslogbot‎ 594,889 bytes +5,544‎ springle Synchronized wmf-config/db-eqiad.php: repool db1037; depool db1030 (duration: 00m 13s) (logmsgbot)

curprev 14:5914:59, 12 July 2015‎ imported>Labslogbot‎ 589,345 bytes +1,013‎ upgraded most packages on sodium (bblack)

curprev 19:4819:48, 11 July 2015‎ imported>Labslogbot‎ 588,332 bytes +1,475‎ stopping labsdb1002 after table corruption has been detected (jynus)
curprev 00:4600:46, 11 July 2015‎ imported>Labslogbot‎ 586,857 bytes +2,703‎ Upgraded Elasticsearch to 1.6.0 on logstash1005; replicas recovering now (bd808)

curprev 00:3900:39, 10 July 2015‎ imported>Labslogbot‎ 584,154 bytes +5,464‎ starting restbase1004 (urandom)

curprev 01:0701:07, 9 July 2015‎ imported>Labslogbot‎ 578,690 bytes +4,522‎ uranium - deleted apache logs older than 90 days (mutante)

curprev 23:5423:54, 7 July 2015‎ imported>Labslogbot‎ 574,168 bytes +2,953‎ kafka brokers 1018 & 1021 were demoted; i have triggered a leader election and they are leaders again (jgage)
curprev 01:1201:12, 7 July 2015‎ imported>Labslogbot‎ 571,215 bytes +5,225‎ Re-pooled mw1152 at 20:46 UTC, did not log it then. (ori)

curprev 22:3022:30, 5 July 2015‎ imported>Labslogbot‎ 565,990 bytes +1,663‎ Restarted logstash on logstah1001; Hung due to OOM errors (bd808)

curprev 23:4923:49, 4 July 2015‎ imported>Labslogbot‎ 564,327 bytes +985‎ Ran "mwscript updateSpecialPages.php labswiki --override --only=Wantedpages" on silver, completed in 0.44 seconds (Krenair)
curprev 01:0001:00, 4 July 2015‎ imported>Labslogbot‎ 563,342 bytes +6,467‎ reload haproxy dbproxy1004 (springle)

curprev 22:3422:34, 2 July 2015‎ imported>Labslogbot‎ 556,875 bytes +3,571‎ legoktm Synchronized php-1.26wmf12/extensions/CentralAuth/: Made use of new USE_MULTI_COMMIT flag in user merge jobs (duration: 00m 18s) (logmsgbot)
curprev 00:4400:44, 2 July 2015‎ imported>Labslogbot‎ 553,304 bytes +3,706‎ Repooling mw1152 (HHVM image scaler) for testing) (ori)

curprev 01:0301:03, 1 July 2015‎ imported>Labslogbot‎ 549,598 bytes +6,042‎ Disabling Puppet on mw1152 for 12h to hack apache config to log locally (ori)

curprev 00:3700:37, 30 June 2015‎ imported>Labslogbot‎ 543,556 bytes +4,176‎ restbase1* upgrade to cassandra 2.1.7 completed (godog)

curprev 00:5200:52, 29 June 2015‎ imported>Labslogbot‎ 539,380 bytes +718‎ restart eventlogging auto-purge on m4 (springle)

curprev 23:3023:30, 27 June 2015‎ imported>Labslogbot‎ 538,662 bytes +510‎ Deleted corrupt shards on logstash1004 and logstash1005. Recovery in process (bd808)

curprev 23:5723:57, 26 June 2015‎ imported>Labslogbot‎ 538,152 bytes +2,020‎ Logstash log ingestion working again after forcing recovery of replicas for logstash-2015.06.26; new logs were being rejected with only a primary shard available (bd808)
curprev 00:5100:51, 26 June 2015‎ imported>Labslogbot‎ 536,132 bytes +6,360‎ reverted restbase1001 canary to 90817c2a (gwicke)

curprev 00:4300:43, 25 June 2015‎ imported>Labslogbot‎ 529,772 bytes +4,757‎ update crm from bd8a00196071ddd04efbff7b30567dd9357c9000 to e923225e423948bd70440e2d1131460b10cefac1 (awight)

curprev 01:0101:01, 24 June 2015‎ imported>Labslogbot‎ 525,015 bytes +7,464‎ rolling restart of cassandra instances to rule out a single node in funky state causing elevated p99 latency (gwicke)

curprev 01:1701:17, 23 June 2015‎ imported>Labslogbot‎ 517,551 bytes +2,806‎ krinkle Synchronized docroot and w: (no message) (duration: 00m 12s) (logmsgbot)

curprev 16:2616:26, 22 June 2015‎ imported>Labslogbot‎ 514,745 bytes +514,745‎ jynus Synchronized wmf-config/db-eqiad.php: Repool es1001, depool es1002 (duration: 00m 14s) (logmsgbot)

Server Admin Log: Revision history

26 February 2016

25 February 2016

24 February 2016

23 February 2016

21 February 2016

20 February 2016

19 February 2016

18 February 2016

17 February 2016

15 February 2016

14 February 2016

13 February 2016

12 February 2016

11 February 2016

10 February 2016

9 February 2016

7 February 2016

6 February 2016

5 February 2016

4 February 2016

2 February 2016

1 February 2016

30 January 2016

29 January 2016

28 January 2016

27 January 2016

26 January 2016

24 January 2016

23 January 2016

22 January 2016

21 January 2016

20 January 2016

18 January 2016

17 January 2016

16 January 2016

15 January 2016

14 January 2016

13 January 2016

12 January 2016

11 January 2016

9 January 2016

8 January 2016

7 January 2016

6 January 2016

5 January 2016

4 January 2016

3 January 2016

2 January 2016

31 December 2015

30 December 2015

29 December 2015

27 December 2015

26 December 2015

25 December 2015

24 December 2015

23 December 2015

22 December 2015

20 December 2015

19 December 2015

18 December 2015

17 December 2015

16 December 2015

15 December 2015

13 December 2015

12 December 2015

11 December 2015

10 December 2015

9 December 2015

8 December 2015

6 December 2015

5 December 2015

4 December 2015

3 December 2015

2 December 2015

1 December 2015

30 November 2015

28 November 2015

27 November 2015

26 November 2015