You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Server Admin Log"

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | older 100) (20 | 50 | 100 | 250 | 500)
  • curprev 01:13, 29 September 2015imported>Labslogbot 866,210 bytes +2,479 didnt powercycle analytics1035 yet, it recovered on its own. (robh)
  • curprev 23:17, 27 September 2015imported>Labslogbot 863,731 bytes +851 ori@tin Synchronized php-1.26wmf24/./resources/src/mediawiki.toolbar/toolbar.less: I94ced06178: mediawiki.toolbar: temporary workaround for T113868 (duration: 00m 17s) (logmsgbot)
  • curprev 19:04, 26 September 2015imported>Labslogbot 862,880 bytes +189 restarting Jenkins. Just in case :-D (hashar)
  • curprev 23:54, 25 September 2015imported>Labslogbot 862,691 bytes +2,772 ori@tin Synchronized php-1.26wmf24/includes/GlobalFunctions.php: 37c6972f94: Made wfIsBadImage() use APC (duration: 00m 17s) (logmsgbot)
  • curprev 01:28, 25 September 2015imported>Labslogbot 859,919 bytes +5,372 krinkle@tin Synchronized php-1.26wmf24/extensions/WikimediaEvents: I5608f8ffd1c - Fix trailing comma (duration: 00m 17s) (logmsgbot)
  • curprev 01:15, 24 September 2015imported>Labslogbot 854,547 bytes +4,662 ori@tin Synchronized php-1.26wmf24/includes: Ifa0d4cfe8e3: Backport I1ff61153d and I8e4c3d5a5 (duration: 00m 23s) (logmsgbot)
  • curprev 00:56, 23 September 2015imported>Labslogbot 849,885 bytes +4,192 legoktm@tin Synchronized php-1.26wmf24/extensions/Echo/Hooks.php: Remove duplicate 'MediaWiki' prefix from echo.unseen stats (duration: 00m 12s) (logmsgbot)
  • curprev 00:32, 22 September 2015imported>Labslogbot 845,693 bytes +4,741 Disabled Puppet for 24h on hafnium and stopped ganglia-monitor. gmond was saturating CPU. (ori)
  • curprev 22:34, 20 September 2015imported>Labslogbot 840,952 bytes +735 reloda pybal on lvs1012 (yuvipanda)
  • curprev 23:12, 19 September 2015imported>Labslogbot 840,217 bytes +1,606 begining Cassandra repair on restbase1005 (nodetool repair -pr) (urandom)
  • curprev 23:22, 18 September 2015imported>Labslogbot 838,611 bytes +4,102 update fundraising-tools from 3e0e3ae799a507b378d0ece3e71631b10b361329 to e1b60fa2c258fd4ff55905b03a4d8886132278c1 (awight)
  • curprev 00:14, 18 September 2015imported>Labslogbot 834,509 bytes +6,909 restarted logstash on logstash1001 (ori)
  • curprev 00:35, 17 September 2015imported>Labslogbot 827,600 bytes +6,029 updated payments from 155cdeb737c01baf62551292764fd2f5a93a9a63 to 1bdd287b083032ff418434ad6bb6920735af918a (cwdent)
  • curprev 01:03, 16 September 2015imported>Labslogbot 821,571 bytes +6,779 krinkle@tin Synchronized php-1.26wmf23/resources/src/mediawiki/mediawiki.js: hotfix Ia2fcd13f4 (duration: 00m 12s) (logmsgbot)
  • curprev 00:50, 15 September 2015imported>Labslogbot 814,792 bytes +4,143 reinstalling lvs300[34] to jessie (bblack)
  • curprev 06:02, 13 September 2015imported>Labslogbot 810,649 bytes +363 l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 13 06:02:52 UTC 2015 (duration 2m 51s) (logmsgbot)
  • curprev 20:15, 12 September 2015imported>Labslogbot 810,286 bytes +677 Rolling back Echo to 1.26wmf21 branch on mw1017 (testwiki) to measure increase in render-blocking CSS size (ori)
  • curprev 21:21, 11 September 2015imported>Labslogbot 809,609 bytes +1,756 shutdown nodepool on labnodepool1001.eqiad.wmnet until monday (hashar)
  • curprev 01:16, 11 September 2015imported>Labslogbot 807,853 bytes +4,964 ori@tin Synchronized php-1.26wmf22/extensions/TitleBlacklist: 9bf13dbe0b, 3203b045f7 (duration: 00m 12s) (logmsgbot)
  • curprev 01:04, 10 September 2015imported>Labslogbot 802,889 bytes +5,413 ori@tin Synchronized php-1.26wmf21/extensions/NavigationTiming: I2605c746b: Ensure timings are reported after the page has loaded (duration: 00m 13s) (logmsgbot)
  • curprev 00:00, 9 September 2015imported>Labslogbot 797,476 bytes +4,248 catrope@tin Finished scap: Need to update i18n for a new Echo message (duration: 23m 08s) (logmsgbot)
  • curprev 00:46, 8 September 2015imported>Labslogbot 793,228 bytes +1,579 mwscript deleteEqualMessages.php --wiki eswiki (Krinkle)
  • curprev 04:28, 6 September 2015imported>Labslogbot 791,649 bytes +364 l10nupdate@tin ResourceLoader cache refresh completed at Sun Sep 6 04:27:57 UTC 2015 (duration 27m 56s) (logmsgbot)
  • curprev 23:37, 5 September 2015imported>Labslogbot 791,285 bytes +434 mwscript deleteEqualMessages.php --wiki fywiktionary (Krinkle)
  • curprev 23:52, 4 September 2015imported>Labslogbot 790,851 bytes +2,618 mattflaschen@tin Synchronized wmf-config/InitialiseSettings-labs.php: Beta-only change (duration: 00m 12s) (logmsgbot)
  • curprev 00:11, 4 September 2015imported>Labslogbot 788,233 bytes +6,425 krinkle@tin Synchronized php-1.26wmf21/includes/resourceloader/ResourceLoader.php: I24f68e34a9fa4918 (duration: 00m 12s) (logmsgbot)
  • curprev 00:36, 3 September 2015imported>Labslogbot 781,808 bytes +5,582 ori@tin Synchronized php-1.26wmf21/includes/parser/Preprocessor_Hash.php: Idd1acd903: Decline to cache preprocessor items larger than 1 Mb (duration: 00m 11s) (logmsgbot)
  • curprev 00:33, 2 September 2015imported>Labslogbot 776,226 bytes +4,769 krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/235366/ (duration: 00m 13s) (logmsgbot)
  • curprev 23:56, 31 August 2015imported>Labslogbot 771,457 bytes +4,638 krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/233665/ (duration: 00m 11s) (logmsgbot)
  • curprev 12:58, 30 August 2015imported>Labslogbot 766,819 bytes +1,020 lvchange -ay labstore/others on labstore1002 (godog)
  • curprev 15:26, 29 August 2015imported>Labslogbot 765,799 bytes +863 killing idle mysql connections from phabricator and setting wait and interactive timeout to 60 (jynus)
  • curprev 23:45, 28 August 2015imported>Labslogbot 764,936 bytes +2,516 krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/234679/ (duration: 06m 56s) (logmsgbot)
  • curprev 00:24, 28 August 2015imported>Labslogbot 762,420 bytes +5,788 powercycled mw2027 (mutante)
  • curprev 00:31, 27 August 2015imported>Labslogbot 756,632 bytes +6,046 finished phabricator upgrade, everything appears to be working (twentyafterfour)
  • curprev 00:28, 26 August 2015imported>Labslogbot 750,586 bytes +5,150 ori@tin Synchronized php-1.26wmf20/extensions/Scribunto/engines/LuaCommon/LuaCommon.php: (no message) (duration: 00m 13s) (logmsgbot)
  • curprev 23:46, 24 August 2015imported>Labslogbot 745,436 bytes +3,386 mattflaschen@tin Synchronized wmf-config: Remove wgFlowOccupyPages (duration: 00m 12s) (logmsgbot)
  • curprev 16:54, 23 August 2015imported>Labslogbot 742,050 bytes +219 bouncing Cassandra on restbase1001 to apply temporary GC settings (urandom)
  • curprev 23:08, 22 August 2015imported>Labslogbot 741,831 bytes +953 krenair@tin Synchronized php-1.26wmf19/extensions/AbuseFilter/maintenance/addMissingLoggingEntries.php: (no message) (duration: 01m 05s) (logmsgbot)
  • curprev 00:26, 22 August 2015imported>Labslogbot 740,878 bytes +1,894 deleting blog.sh and blog_pageviews crontab from stat1003 (mutante)
  • curprev 23:40, 20 August 2015imported>Labslogbot 738,984 bytes +4,846 ebernhardson@tin Synchronized php-1.26wmf19/extensions/CirrusSearch/: Fix some cirrussearch logspam (duration: 00m 13s) (logmsgbot)
  • curprev 21:20, 19 August 2015imported>Labslogbot 734,138 bytes +4,027 livehack reverted sodium back to normal, testing done (robh)
  • curprev 01:11, 19 August 2015imported>Labslogbot 730,111 bytes +3,199 updated kartotherian (yurik)
  • curprev 00:02, 18 August 2015imported>Labslogbot 726,912 bytes +3,254 analytics1041 down, attempting power cycle (ottomata)
  • curprev 18:15, 16 August 2015imported>Labslogbot 723,658 bytes +694 krenair@tin Synchronized wmf-config/extension-list: https://gerrit.wikimedia.org/r/#/c/169612/ - remove Extension:Oversight (duration: 00m 21s) (logmsgbot)
  • curprev 22:56, 15 August 2015imported>Labslogbot 722,964 bytes +961 removed 13 bounce_records for User:odder from bouncehandler database (legoktm)
  • curprev 01:15, 15 August 2015imported>Labslogbot 722,003 bytes +2,802 starting broker on analytics1012, camus wasn't happy about that either. hrm. (ottomata)
  • curprev 23:45, 13 August 2015imported>Labslogbot 719,201 bytes +6,324 rollback paymentswiki from 2e7b449224317779d53ff84527166c0d378a0a40 to 823393264d6795bbaec490ff86f17580f722e598 (awight)
  • curprev 01:26, 13 August 2015imported>Labslogbot 712,877 bytes +3,637 Restarted conversion of support desk from LQT->Flow using convertLqtPageOnLocalWiki.php, using hhvm on mw1041 (matt_flaschen)
  • curprev 01:22, 12 August 2015imported>Labslogbot 709,240 bytes +5,841 zirconium - shut down, i'm sure, mollyguard (mutante)
  • curprev 01:20, 11 August 2015imported>Labslogbot 703,399 bytes +4,246 ori@tin Synchronized php-1.26wmf17/includes/resourceloader/ResourceLoader.php: I2089b21fc: Revert resourceloader: Add must-revalidate to Cache-Control (duration: 00m 12s) (logmsgbot)
  • curprev 17:57, 9 August 2015imported>Labslogbot 699,153 bytes +783 issuing nodetool cleanup on restbase1006 (urandom)
  • curprev 14:58, 8 August 2015imported>Labslogbot 698,370 bytes +1,003 issuing nodetool cleanup on restbase1005 (urandom)
  • curprev 21:00, 7 August 2015imported>Labslogbot 697,367 bytes +2,551 krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/230227/ - should be a noop for prod (duration: 00m 12s) (logmsgbot)
  • curprev 01:08, 7 August 2015imported>Labslogbot 694,816 bytes +7,694 ori Synchronized php-1.26wmf17/extensions/Gadgets: I2089b21fc: Updated mediawiki/core Project: mediawiki/extensions/Gadgets (duration: 00m 13s) (logmsgbot)
  • curprev 00:49, 6 August 2015imported>Labslogbot 687,122 bytes +7,735 reset password for User:Tonval after identify verification (Jamesofur)
  • curprev 23:59, 4 August 2015imported>Labslogbot 679,387 bytes +4,864 maxsem Synchronized php-1.26wmf16/extensions/WikimediaEvents/: SWAT (duration: 00m 12s) (logmsgbot)
  • curprev 00:02, 4 August 2015imported>Labslogbot 674,523 bytes +6,039 updated paymentswiki to a8c0ecbedef6179c78ed833da9f2049cb0f2641b (awight)
  • curprev 00:03, 3 August 2015imported>Labslogbot 668,484 bytes +789 krenair Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/228198/ (duration: 00m 12s) (logmsgbot)
  • curprev 06:05, 1 August 2015imported>Labslogbot 667,695 bytes +491 removing some old apache access logs from mw1114 (_joe_)
  • curprev 00:12, 1 August 2015imported>Labslogbot 667,204 bytes +3,845 ori Synchronized extract2.php: Ie919881a4: Add an API listing template to the allowed templates in extract2.php (logmsgbot)
  • curprev 00:35, 31 July 2015imported>Labslogbot 663,359 bytes +7,505 catrope Synchronized php-1.26wmf16/extensions/Flow/includes/Model/WikiReference.php: debugging (duration: 00m 12s) (logmsgbot)
  • curprev 00:48, 30 July 2015imported>Labslogbot 655,854 bytes +7,841 ori Synchronized php-1.26wmf15/includes/Message.php: 160f69871c: Debug logging for T102199 (duration: 00m 13s) (logmsgbot)
  • curprev 00:43, 29 July 2015imported>Labslogbot 648,013 bytes +4,914 ori Synchronized php-1.26wmf15/extensions/AbuseFilter: Revert "Revert "Conversion to using getMainStashInstance()"" (duration: 00m 12s) (logmsgbot)
  • curprev 01:11, 28 July 2015imported>Labslogbot 643,099 bytes +5,843 krenair Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/227371/ (duration: 00m 11s) (logmsgbot)
  • curprev 01:18, 27 July 2015imported>Labslogbot 637,256 bytes +1,228 Re-pooling mw1159 and mw1160; ran out of time for debugging. (ori)
  • curprev 20:51, 25 July 2015imported>Labslogbot 636,028 bytes +1,141 rolling restart of restbase instances (gwicke)
  • curprev 21:57, 24 July 2015imported>Labslogbot 634,887 bytes +2,829 running mwscript populateContentModel.php --wiki=enwiki --ns=all --table=page (legoktm)
  • curprev 00:21, 24 July 2015imported>Labslogbot 632,058 bytes +5,446 Re-enabled Puppet on mw1153 (ori)
  • curprev 01:05, 23 July 2015imported>Labslogbot 626,612 bytes +6,733 phab is back (twentyafterfour)
  • curprev 23:45, 21 July 2015imported>Labslogbot 619,879 bytes +4,786 catrope Synchronized wmf-config/InitialiseSettings.php: Set $wgVectorResponsive = true on testwiki (duration: 00m 12s) (logmsgbot)
  • curprev 23:44, 20 July 2015imported>Labslogbot 615,093 bytes +1,285 removed experimental nodes (1008, 1009) from system.peers on production C* nodes (gwicke)
  • curprev 00:02, 20 July 2015imported>Labslogbot 613,808 bytes +1,946 DNS update - adding language "azb" to langlist (mutante)
  • curprev 20:58, 18 July 2015imported>Labslogbot 611,862 bytes +821 legoktm Synchronized wmf-config/InitialiseSettings-labs.php: labs only (duration: 00m 12s) (logmsgbot)
  • curprev 00:49, 18 July 2015imported>Labslogbot 611,041 bytes +2,129 restored recurring globalcollect batch size of 250 (ejegg)
  • curprev 21:27, 16 July 2015imported>Labslogbot 608,912 bytes +3,677 bounced nutcracker on mw1139 as well. hashar noticed flood of errors from these hosts on https://logstash.wikimedia.org/#/dashboard/elasticsearch/mediawiki-errors . lack of monitoring / alerts is troubling. (ori)
  • curprev 01:22, 16 July 2015imported>Labslogbot 605,235 bytes +7,251 es1.6 upgrade: upgrade elastic1023 (manybubbles)
  • curprev 23:46, 14 July 2015imported>Labslogbot 597,984 bytes +3,095 es1.6 upgrade: upgraded elastic1011 (manybubbles)
  • curprev 01:22, 14 July 2015imported>Labslogbot 594,889 bytes +5,544 springle Synchronized wmf-config/db-eqiad.php: repool db1037; depool db1030 (duration: 00m 13s) (logmsgbot)
  • curprev 14:59, 12 July 2015imported>Labslogbot 589,345 bytes +1,013 upgraded most packages on sodium (bblack)
  • curprev 19:48, 11 July 2015imported>Labslogbot 588,332 bytes +1,475 stopping labsdb1002 after table corruption has been detected (jynus)
  • curprev 00:46, 11 July 2015imported>Labslogbot 586,857 bytes +2,703 Upgraded Elasticsearch to 1.6.0 on logstash1005; replicas recovering now (bd808)
  • curprev 00:39, 10 July 2015imported>Labslogbot 584,154 bytes +5,464 starting restbase1004 (urandom)
  • curprev 01:07, 9 July 2015imported>Labslogbot 578,690 bytes +4,522 uranium - deleted apache logs older than 90 days (mutante)
  • curprev 23:54, 7 July 2015imported>Labslogbot 574,168 bytes +2,953 kafka brokers 1018 & 1021 were demoted; i have triggered a leader election and they are leaders again (jgage)
  • curprev 01:12, 7 July 2015imported>Labslogbot 571,215 bytes +5,225 Re-pooled mw1152 at 20:46 UTC, did not log it then. (ori)
  • curprev 22:30, 5 July 2015imported>Labslogbot 565,990 bytes +1,663 Restarted logstash on logstah1001; Hung due to OOM errors (bd808)
  • curprev 23:49, 4 July 2015imported>Labslogbot 564,327 bytes +985 Ran "mwscript updateSpecialPages.php labswiki --override --only=Wantedpages" on silver, completed in 0.44 seconds (Krenair)
  • curprev 01:00, 4 July 2015imported>Labslogbot 563,342 bytes +6,467 reload haproxy dbproxy1004 (springle)
  • curprev 22:34, 2 July 2015imported>Labslogbot 556,875 bytes +3,571 legoktm Synchronized php-1.26wmf12/extensions/CentralAuth/: Made use of new USE_MULTI_COMMIT flag in user merge jobs (duration: 00m 18s) (logmsgbot)
  • curprev 00:44, 2 July 2015imported>Labslogbot 553,304 bytes +3,706 Repooling mw1152 (HHVM image scaler) for testing) (ori)
  • curprev 01:03, 1 July 2015imported>Labslogbot 549,598 bytes +6,042 Disabling Puppet on mw1152 for 12h to hack apache config to log locally (ori)
  • curprev 00:37, 30 June 2015imported>Labslogbot 543,556 bytes +4,176 restbase1* upgrade to cassandra 2.1.7 completed (godog)
  • curprev 00:52, 29 June 2015imported>Labslogbot 539,380 bytes +718 restart eventlogging auto-purge on m4 (springle)
  • curprev 23:30, 27 June 2015imported>Labslogbot 538,662 bytes +510 Deleted corrupt shards on logstash1004 and logstash1005. Recovery in process (bd808)
  • curprev 23:57, 26 June 2015imported>Labslogbot 538,152 bytes +2,020 Logstash log ingestion working again after forcing recovery of replicas for logstash-2015.06.26; new logs were being rejected with only a primary shard available (bd808)
  • curprev 00:51, 26 June 2015imported>Labslogbot 536,132 bytes +6,360 reverted restbase1001 canary to 90817c2a (gwicke)
  • curprev 00:43, 25 June 2015imported>Labslogbot 529,772 bytes +4,757 update crm from bd8a00196071ddd04efbff7b30567dd9357c9000 to e923225e423948bd70440e2d1131460b10cefac1 (awight)
  • curprev 01:01, 24 June 2015imported>Labslogbot 525,015 bytes +7,464 rolling restart of cassandra instances to rule out a single node in funky state causing elevated p99 latency (gwicke)
  • curprev 01:17, 23 June 2015imported>Labslogbot 517,551 bytes +2,806 krinkle Synchronized docroot and w: (no message) (duration: 00m 12s) (logmsgbot)
  • curprev 16:26, 22 June 2015imported>Labslogbot 514,745 bytes +514,745 jynus Synchronized wmf-config/db-eqiad.php: Repool es1001, depool es1002 (duration: 00m 14s) (logmsgbot)
(newest | oldest) View ( | older 100) (20 | 50 | 100 | 250 | 500)