You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Release Engineering/SAL: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

1 October 2016

  • curprev 09:4109:41, 1 October 2016imported>Labslogbot 132,886 bytes +137 beta: shutdown deployment-db1 and deployment-db2 . Databases have been migrated to other hosts T138778 (hashar)

29 September 2016

  • curprev 15:4315:43, 29 September 2016imported>Labslogbot 132,749 bytes +641 logstash-beta: refreshed the field list via https://logstash-beta.wmflabs.org/app/kibana#/settings/indices/logstash (hashar)

28 September 2016

  • curprev 23:5623:56, 28 September 2016imported>Labslogbot 132,108 bytes +1,698 Deleted varnish cache files on deployment-cache-upload04 to free up space, disk full (MaxSem)

27 September 2016

  • curprev 20:1120:11, 27 September 2016imported>Labslogbot 130,410 bytes +195 Reloading Zuul to deploy 3c3289aa1a for T143938 and T146783 (hashar)

26 September 2016

24 September 2016

  • curprev 20:0820:08, 24 September 2016imported>Labslogbot 129,651 bytes +400 deployment-tin is shutdown. Replaced by Jessie deployment-tin02 (hashar)

23 September 2016

  • curprev 20:2120:21, 23 September 2016imported>Labslogbot 129,251 bytes +686 integration: salt -v '*trusty*' cmd.run 'service mysql start' (hashar)

22 September 2016

  • curprev 19:2919:29, 22 September 2016imported>Labslogbot 128,565 bytes +241 switching Jenkins slaves workspace from /mnt/jenkins-workspace to /srv/jenkins-workspace (actually the same dir/inode on the filesystem) (hasharAway)

21 September 2016

20 September 2016

  • curprev 21:4921:49, 20 September 2016imported>Labslogbot 126,058 bytes +2,278 Deleting deployment-mira02 /srv was too small. Replaced by deployment-mira (hashar)

19 September 2016

17 September 2016

16 September 2016

  • curprev 21:0321:03, 16 September 2016imported>Labslogbot 122,412 bytes +1,025 deployment-tin did a git gc on /srv/deployment/ores That freed up disk space and cleared an alarm on co master mira02 (hashar)

15 September 2016

  • curprev 22:2922:29, 15 September 2016imported>Labslogbot 121,387 bytes +1,969 sudo salt '*precise*' cmd.run 'service mysql start', all mysql's are down (legoktm)

14 September 2016

  • curprev 21:3721:37, 14 September 2016imported>Labslogbot 119,418 bytes +537 integration: setting "ulimit -c 2097152" on all slaves due to Zend PHP segfaulting T142158 (hashar)

13 September 2016

  • curprev 17:0217:02, 13 September 2016imported>Labslogbot 118,881 bytes +1,525 re-enabling beta cluster jenkins jobs following maintenance window (marxarelli)

12 September 2016

  • curprev 14:4114:41, 12 September 2016imported>Labslogbot 117,356 bytes +588 applied base::firewall, beta::deployaccess, mediawiki::conftool, role::mediawiki::appserver to deployment-mediawiki04.deployment-prep.eqiad.wmflabs (Debian jessie instance) - T144006 (elukey)

11 September 2016

9 September 2016

8 September 2016

  • curprev 21:3321:33, 8 September 2016imported>Labslogbot 116,286 bytes +1,462 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/309413 " Inject PHP_BIN=php5 for php53 jobs" (hashar)

7 September 2016

6 September 2016

5 September 2016

  • curprev 14:0814:08, 5 September 2016imported>Labslogbot 112,066 bytes +130 Refreshing Nodepool base images for Trusty and Jessie. Managed to build new ones after T143769 (hashar)

2 September 2016

1 September 2016

  • curprev 19:0019:00, 1 September 2016imported>Labslogbot 111,759 bytes +670 T130861: Restarting Cassandra on deployment-restbase0[1-2] (urandom)

31 August 2016

  • curprev 23:4023:40, 31 August 2016imported>Labslogbot 111,089 bytes +986 forced puppet run on deployment-salt02. Had not run automatically for 8 hours (bd808)

30 August 2016

  • curprev 23:3123:31, 30 August 2016imported>Labslogbot 110,103 bytes +2,327 cherry-picking https://gerrit.wikimedia.org/r/#/c/307656/ fixed puppet on the elasticsearch machines! (yuvipanda)

29 August 2016

  • curprev 07:5607:56, 29 August 2016imported>Labslogbot 107,776 bytes +226 hard rebooting integration-slave-trusty-1012 via horizon and restarting puppet manually (hashar)

27 August 2016

  • curprev 20:5120:51, 27 August 2016imported>Labslogbot 107,550 bytes +412 integration: tweak sudo policy for jenkins-deploy running cowbuilder: env_keep+=DEB_BUILD_OPTIONS (hashar)

26 August 2016

  • curprev 22:2922:29, 26 August 2016imported>Labslogbot 107,138 bytes +551 deploying https://gerrit.wikimedia.org/r/307025 (legoktm)
  • curprev 00:2800:28, 26 August 2016imported>Labslogbot 106,587 bytes +795 restarted puppetmaster service on deployment-puppetmaster (twentyafterfour)

24 August 2016

23 August 2016

  • curprev 18:2118:21, 23 August 2016imported>Labslogbot 105,304 bytes +409 deploying https://gerrit.wikimedia.org/r/306257 (legoktm)

22 August 2016

18 August 2016

  • curprev 22:0322:03, 18 August 2016imported>Labslogbot 104,823 bytes +334 deployment-fluorine02: Hack 'datasets:x:10003:997::/home/datasets:/bin/bash' into /etc/passwd for T117028 (bd808)

17 August 2016

  • curprev 22:2822:28, 17 August 2016imported>Labslogbot 104,489 bytes +524 deploying https://gerrit.wikimedia.org/r/305408 (legoktm)

16 August 2016

  • curprev 23:1023:10, 16 August 2016imported>Labslogbot 103,965 bytes +350 max_servers at 6, seeing 6 allocated instances, still seeing 403 already used 10 of 10 instances :(( (thcipriani)

15 August 2016

  • curprev 21:3021:30, 15 August 2016imported>Labslogbot 103,615 bytes +710 update scap on beta to 3.2.3-1 bugfix release (thcipriani)

13 August 2016

10 August 2016

  • curprev 23:5623:56, 10 August 2016imported>Labslogbot 102,533 bytes +347 deploying https://gerrit.wikimedia.org/r/304149 (legoktm)

9 August 2016

  • curprev 23:2123:21, 9 August 2016imported>Labslogbot 102,186 bytes +412 ladsgroup@deployment-sca03:~$ sudo service celery-ores-worker restart (Amir1)
  • curprev 00:5600:56, 9 August 2016imported>Labslogbot 101,774 bytes +341 deploying https://gerrit.wikimedia.org/r/303726 (legoktm)

7 August 2016

  • curprev 12:0112:01, 7 August 2016imported>Labslogbot 101,433 bytes +604 Nodepool: can't spawn instances due to: Forbidden: Quota exceeded for instances: Requested 1, but already used 10 of 10 instances (HTTP 403) (hashar)

6 August 2016

  • curprev 12:3112:31, 6 August 2016imported>Labslogbot 100,829 bytes +240 restarting uwsgi-ores and celery-ores-worker in deployment-sca03 (Amir1)

5 August 2016

  • curprev 17:5417:54, 5 August 2016imported>Labslogbot 100,589 bytes +251 Cherry-picked https://gerrit.wikimedia.org/r/#/c/299825/3 for testing (bd808)
  • curprev 00:4100:41, 5 August 2016imported>Labslogbot 100,338 bytes +421 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/303113 (Krinkle)

1 August 2016

  • curprev 20:2820:28, 1 August 2016imported>Labslogbot 99,917 bytes +170 restarting deployment-ms-be01, not responding to ssh, mw-fe01 requests timing out (thcipriani)

29 July 2016

  • curprev 23:2723:27, 29 July 2016imported>Labslogbot 99,747 bytes +842 Rebooting deployment-logstash2; Console showed hung task timeouts (P3606) (bd808)

28 July 2016

  • curprev 21:4621:46, 28 July 2016imported>Labslogbot 98,905 bytes +580 xintegration: change sudo policy for jenkins-deploy to help on T141538 : env_keep+=WORKSPACE (hashar_)

27 July 2016

  • curprev 16:1316:13, 27 July 2016imported>Labslogbot 98,325 bytes +878 salt -v '*slave-trusty*' cmd.run 'service mysql start' ( was missing on integration-slave-trusty-1011.integration.eqiad.wmflabs ) (hashar)

26 July 2016

  • curprev 20:1320:13, 26 July 2016imported>Labslogbot 97,447 bytes +795 Zuul deployed https://gerrit.wikimedia.org/r/301093 which adds 'debian-glue' job on all of operations/debs/ repos (hashar)
  • curprev 01:0801:08, 26 July 2016imported>Labslogbot 96,652 bytes +341 deployed ores a291da1 in sca03, ores-beta.wmflabs.org works as expected (Amir1)

23 July 2016

  • curprev 20:0620:06, 23 July 2016imported>Labslogbot 96,311 bytes +316 Cleanup jobrunner01 logs via -- sudo logrotate --force /etc/logrotate.d/mediawiki_jobrunner (bd808)

22 July 2016

  • curprev 20:2620:26, 22 July 2016imported>Labslogbot 95,995 bytes +1,322 T141114 upgraded jenkins-debian-glue from v0.13.0 to v0.17.0 on integration-slave-jessie-1001 and integration-slave-jessie-1002 (hashar)

21 July 2016

  • curprev 16:1216:12, 21 July 2016imported>Labslogbot 94,673 bytes +640 Updated our JJB fork cherry picking f74501e781f by madhuvishy. Was made to support the maven release plugin. Branch bump is 10f2bcd..6fcaf39 (hashar)
  • curprev 00:1300:13, 21 July 2016imported>Labslogbot 94,033 bytes +1,279 Cherry-picked https://gerrit.wikimedia.org/r/#/c/299825/ to deployment-puppetmaster so wdqs nginx log parsing can be tested (bd808)

18 July 2016

  • curprev 20:5620:56, 18 July 2016imported>Labslogbot 92,754 bytes +385 Deleted deployment-fluorine:/srv/mw-log/archive/*-201605* freed 30 GB (thcipriani)

16 July 2016

15 July 2016

  • curprev 08:3408:34, 15 July 2016imported>Labslogbot 92,317 bytes +132 Unpooling integration-slave-precise-1002 will use it as a zuul-server test instance temporarily (hashar)

14 July 2016

  • curprev 18:5418:54, 14 July 2016imported>Labslogbot 92,185 bytes +316 deployment-prep manually edited elasticsearch.yml on deployment-elastic05 and restarted to get it listening on eth0. Still looking into why puppet wrote out wrong config file (ebernhardson)

13 July 2016

  • curprev 16:0516:05, 13 July 2016imported>Labslogbot 91,869 bytes +512 Installing Cassandra 2.2.6-wmf1 on deployment-restbase0[1-2].deployment-prep.eqiad.wmflabs : T126629 (urandom)

12 July 2016

  • curprev 20:2920:29, 12 July 2016imported>Labslogbot 91,357 bytes +643 integration: force running unattended upgrade on all instances: salt --batch 4 -v '*' cmd.run 'unattended-upgrade' . That upgrades diamond and hhvm among others. imagemagick-common has a prompt though (hashar)

11 July 2016

  • curprev 14:2414:24, 11 July 2016imported>Labslogbot 90,714 bytes +579 Removing ZeroMQ config from the Jenkins jobs. It is now enabled globally. T139923 (hashar)

9 July 2016

  • curprev 18:5718:57, 9 July 2016imported>Labslogbot 90,135 bytes +223 deploying https://gerrit.wikimedia.org/r/297731 and https://gerrit.wikimedia.org/r/298142 (legoktm)

8 July 2016

  • curprev 16:0816:08, 8 July 2016imported>Labslogbot 89,912 bytes +817 scandium: git -C /srv/ssd/zuul/git/mediawiki/services/graphoid remote set-head origin --auto (hashar)

7 July 2016

  • curprev 21:4121:41, 7 July 2016imported>Labslogbot 89,095 bytes +283 Chowned php-master/vendor back to jenkins-deploy (MaxSem)
  • curprev 01:1401:14, 7 July 2016imported>Labslogbot 88,812 bytes +1,085 Restarted logstash on deployment-logstash2 (bd808)

5 July 2016

  • curprev 21:5421:54, 5 July 2016imported>Labslogbot 87,727 bytes +201 CI has drained the gate-and-submit queue (hasharAway)

4 July 2016

  • curprev 18:5818:58, 4 July 2016imported>Labslogbot 87,526 bytes +170 Upgrading arcanist on permanent CI slaves since xhpast was broken T137770 (hashar)

3 July 2016

  • curprev 13:1013:10, 3 July 2016imported>Labslogbot 87,356 bytes +238 phabricator Update phab-01 and phab-05 (phab-02) and phab-03 to fix a security bug in phabricator (Did the update last night but forgot to log it) (paladox)

2 July 2016

  • curprev 13:3813:38, 2 July 2016imported>Labslogbot 87,118 bytes +74 reloading zuul for 15127b2..7e6a2e2 (jzerebecki)

30 June 2016

  • curprev 10:3110:31, 30 June 2016imported>Labslogbot 87,044 bytes +492 Deleting integration-slave-trusty-1015 . Can not bring up mysql T138074 and the ssh slave connection would not hold anyway. Must be broken somehow (hashar)

28 June 2016

27 June 2016

  • curprev 22:3222:32, 27 June 2016imported>Labslogbot 86,456 bytes +680 deployment-prep deployed gerrit.wikimedia.org/r/296279 to puppetmaster to test kibana4 role (eberhardson)

24 June 2016

  • curprev 15:0415:04, 24 June 2016imported>Labslogbot 85,776 bytes +372 switch apps-android-wikimedia-* jobs to Jessie T138506 (hashar)

23 June 2016

  • curprev 13:5813:58, 23 June 2016imported>Labslogbot 85,404 bytes +391 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/295691 (Krinkle)

17 June 2016

  • curprev 10:3510:35, 17 June 2016imported>Labslogbot 85,013 bytes +181 offlined integration-slave-trusty-1015 T138074 (jzerebecki)

14 June 2016

  • curprev 14:2214:22, 14 June 2016imported>Labslogbot 84,832 bytes +275 T136971 on tin MediaWiki 1.28.0-wmf.6, from 1.28.0-wmf.6, successfully checked out. Applying security patches (hashar)

13 June 2016

  • curprev 21:1121:11, 13 June 2016imported>Labslogbot 84,557 bytes +898 https://integration.wikimedia.org/ci/computer/integration-slave-trusty-1015/ put offline. Jenkins cant ssh / pool it for some reason (hashar)

10 June 2016

  • curprev 15:4915:49, 10 June 2016imported>Labslogbot 83,659 bytes +779 reloading zuul for 8c048fb..272d1ec (jzerebecki)

9 June 2016

  • curprev 18:4918:49, 9 June 2016imported>Labslogbot 82,880 bytes +372 restarting nutcracker on deployment-mediawiki02 (hashar)

8 June 2016

7 June 2016

  • curprev 19:2819:28, 7 June 2016imported>Labslogbot 82,364 bytes +449 Nodepool has troubles spawning instances probably due to on going (?) labs maintenance (hashar)

6 June 2016

  • curprev 19:2619:26, 6 June 2016imported>Labslogbot 81,915 bytes +308 Regenerating Nodepool snapshots for Trusty and Jessie (hasharAway)

4 June 2016

  • curprev 00:0900:09, 4 June 2016imported>Labslogbot 81,607 bytes +562 krinkle@integration-slave-trusty-1017:~$ sudo rm -rf /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/Babel (T86730) (Krinkle)

2 June 2016

  • curprev 12:1012:10, 2 June 2016imported>Labslogbot 81,045 bytes +129 Upgraded Zuul upstream code being 66c8e52..30a433b package is 2.1.0-151-g30a433b-wmf1precise1 (hashar)

1 June 2016

  • curprev 17:4917:49, 1 June 2016imported>Labslogbot 80,916 bytes +888 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/292186 (Krinkle)

31 May 2016

  • curprev 20:2420:24, 31 May 2016imported>Labslogbot 80,028 bytes +101 Reloading zuul to pick up I58f878f3fd19dfa21a46a52464575cb06aacbb22 (bd808)

30 May 2016

  • curprev 18:3918:39, 30 May 2016imported>Labslogbot 79,927 bytes +624 Upgraded our Jenkins Job Builder fork to 1.5.0 + a couple of cherry picks: cd63874...10f2bcd (hashar)

28 May 2016

  • curprev 04:4304:43, 28 May 2016imported>Labslogbot 79,303 bytes +95 depooling integration-slave-trusty-1015 to profile phpunit runs (ori)

27 May 2016

26 May 2016

  • curprev 22:2322:23, 26 May 2016imported>Labslogbot 78,916 bytes +588 salt -v '*trusty*' cmd.run 'puppet agent --disable "Chromium needs to be v49. See T136188"' (hashar)

25 May 2016

  • curprev 22:3822:38, 25 May 2016imported>Labslogbot 78,328 bytes +1,025 running puppet agent manually on sca01 (Amir1)

24 May 2016

  • curprev 07:0307:03, 24 May 2016imported>Labslogbot 77,303 bytes +75 rebooting deployment-tin, can't log in (mobrovac)

23 May 2016

  • curprev 19:3519:35, 23 May 2016imported>Labslogbot 77,228 bytes +1,024 killed all mysqld process on Trusty CI slaves (hashar)

20 May 2016

  • curprev 20:3020:30, 20 May 2016imported>Labslogbot 76,204 bytes +151 Killing https://integration.wikimedia.org/ci/job/mediawiki-extensions-qunit/43608/ which has been running for 5 hours (bd808)

19 May 2016

  • curprev 16:4716:47, 19 May 2016imported>Labslogbot 76,053 bytes +778 deployment-tin jenkins worker seems to be back online after [https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update some prodding] (thcipriani)

18 May 2016

  • curprev 13:1613:16, 18 May 2016imported>Labslogbot 75,275 bytes +413 deploying a05e830 to ores nodes (sca01 and ores-web) (Amir1)

13 May 2016

  • curprev 14:3914:39, 13 May 2016imported>Labslogbot 74,862 bytes +601 remove shadow l10nupdate user from deployment-tin and mira in beta (thcipriani)

12 May 2016

  • curprev 22:5322:53, 12 May 2016imported>Labslogbot 74,261 bytes +86 Started dead mysql on integration-slave-precise-1011 (bd808)

11 May 2016

  • curprev 21:0521:05, 11 May 2016imported>Labslogbot 74,175 bytes +1,452 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/288128 #T134946 (hashar)

10 May 2016

  • curprev 19:1719:17, 10 May 2016imported>Labslogbot 72,723 bytes +1,609 beta / CI purging old Linux kernels: salt -v '*' cmd.run 'dpkg -l|grep ^rc|awk "{ print \$2 }"|grep linux-image|xargs dpkg --purge' (hashar)

9 May 2016

  • curprev 20:5820:58, 9 May 2016imported>Labslogbot 71,114 bytes +890 Unbroke puppet on integration-raita.integration.eqiad.wmflabs . Puppet was blocked because role::ci::raita was no more. Fixed by rebasing https://gerrit.wikimedia.org/r/#/c/208024 T115330 (hashar)

6 May 2016

  • curprev 19:4519:45, 6 May 2016imported>Labslogbot 70,224 bytes +172 Restart cassandra-metrics-collector on deployment-restbase0[1-2] (urandom)

5 May 2016

  • curprev 22:0922:09, 5 May 2016imported>Labslogbot 70,052 bytes +139 Promoted Yurik and Jgirault to sysops on beta enwiki. Through shell because logging in is broken for me. (MaxSem)

4 May 2016

  • curprev 21:2821:28, 4 May 2016imported>Labslogbot 69,913 bytes +1,227 deployed puppet FQDN domain patch for OCG: https://gerrit.wikimedia.org/r/286068 and restarted ocg on deployment-pdf0[12] (cscott)

3 May 2016

  • curprev 23:2123:21, 3 May 2016imported>Labslogbot 68,686 bytes +1,748 Changed "Maximum Number of Retries" for ssh agent launch in jenkins for deployment-tin from "0" to "10" (bd808)

2 May 2016

  • curprev 21:2721:27, 2 May 2016imported>Labslogbot 66,938 bytes +1,284 updated OCG to version b775e612520f9cd4acaea42226bcf34df07439f7 (cscott)

30 April 2016

  • curprev 18:3118:31, 30 April 2016imported>Labslogbot 65,654 bytes +136 deploying d4f63a3 from github.com/wiki-ai/ores-wikimedia-config into targets in beta cluster via scap3 (Amir1)

29 April 2016

28 April 2016

  • curprev 22:0322:03, 28 April 2016imported>Labslogbot 65,189 bytes +1,833 deployment-restbase01 upgrade to 2.2.6 complete : T126629 (urandom)

27 April 2016

  • curprev 23:5723:57, 27 April 2016imported>Labslogbot 63,356 bytes +1,474 nodepool instances running again after an openstack rabbitmq restart by andrewbogott (thcipriani)

26 April 2016

  • curprev 20:4520:45, 26 April 2016imported>Labslogbot 61,882 bytes +875 Regenerating Nodepool Jessie snapshot to include composer and HHVM | T128092 (hashar)
  • curprev 00:4600:46, 26 April 2016imported>Labslogbot 61,007 bytes +1,084 temporary keyholder fix in place in beta (thcipriani)

24 April 2016

22 April 2016

  • curprev 18:1318:13, 22 April 2016imported>Labslogbot 59,858 bytes +148 deploying https://gerrit.wikimedia.org/r/284841 (legoktm)

21 April 2016

  • curprev 19:0719:07, 21 April 2016imported>Labslogbot 59,710 bytes +234 scap version testing should be done, puppet should no longer be disabled on hosts (thcipriani)

20 April 2016

  • curprev 22:2822:28, 20 April 2016imported>Labslogbot 59,476 bytes +541 rolling back scap version in beta, legit failure :( (thcipriani)

17 April 2016

  • curprev 01:0101:01, 17 April 2016imported>Labslogbot 58,935 bytes +223 deploying https://gerrit.wikimedia.org/r/283837 (legoktm)

13 April 2016

  • curprev 01:4801:48, 13 April 2016imported>Labslogbot 58,712 bytes +83 deploying https://gerrit.wikimedia.org/r/282952 (legoktm)

12 April 2016

  • curprev 19:4719:47, 12 April 2016imported>Labslogbot 58,629 bytes +869 Cleaned up large hhbc cache file on deployment-medaiwiki03 via `sudo service hhvm stop; sudo rm /var/cache/hhvm/fcgi.hhbc.sq3; sudo service hhvm start` (bd808)

11 April 2016

8 April 2016

  • curprev 22:4622:46, 8 April 2016imported>Labslogbot 57,302 bytes +563 Created blobs1 table for all wiki DBs on Beta Cluster (matt_flaschen)

7 April 2016

  • curprev 20:1620:16, 7 April 2016imported>Labslogbot 56,739 bytes +217 deployment-mediawiki02.deployment-prep.eqiad.wmflabs , cleared up random left over stuff / big logs etc (hashar)

5 April 2016

  • curprev 23:5623:56, 5 April 2016imported>Labslogbot 56,522 bytes +561 Removed cherry-pick and rebased /var/lib/git/operations/puppet on integration-puppetmaster after merge of https://gerrit.wikimedia.org/r/#/c/281706/ (marxarelli)

4 April 2016

  • curprev 17:3017:30, 4 April 2016imported>Labslogbot 55,961 bytes +400 Phabricator going down in about 10 minutes to hopefully address the overheating issue: T131742 (greg-g)

3 April 2016

  • curprev 07:0207:02, 3 April 2016imported>Labslogbot 55,561 bytes +157 deploying https://gerrit.wikimedia.org/r/281079 (legoktm)

2 April 2016

  • curprev 22:5822:58, 2 April 2016imported>Labslogbot 55,404 bytes +425 added local hack to pupetmaster to make scap3 provider more verbose (Amir1)

1 April 2016

  • curprev 22:1622:16, 1 April 2016imported>Labslogbot 54,979 bytes +960 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/281046 (Krinkle)
  • curprev 00:1600:16, 1 April 2016imported>Labslogbot 54,019 bytes +413 created oathauth_users table on centralauth db in beta (csteipp)

30 March 2016

  • curprev 19:3219:32, 30 March 2016imported>Labslogbot 53,606 bytes +709 deleted some nutcracker and hhvm log files on deployment-mediawiki01 to free space (twentyafterfour)

29 March 2016

  • curprev 23:2223:22, 29 March 2016imported>Labslogbot 52,897 bytes +365 running jenkins-jobs update config/ 'mwext-donationinterfacecore125-testextension-zend53' to deploy https://gerrit.wikimedia.org/r/#/c/280261/ (marxarelli)

26 March 2016

25 March 2016

  • curprev 23:5523:55, 25 March 2016imported>Labslogbot 52,458 bytes +1,083 deleting instances integration-slave-trusty-1002 and integration-slave-trusty-1005 (marxarelli)

24 March 2016

  • curprev 16:3916:39, 24 March 2016imported>Labslogbot 51,375 bytes +350 restarted rsync service on deployment-tin (thcipriani)

23 March 2016

  • curprev 23:0023:00, 23 March 2016imported>Labslogbot 51,025 bytes +200 rm-rf integration-slave-trusty-1013:/mnt/home/jenkins-deploy/tmpfs/jenkins-2/karma-54925082/ (bad permissions, caused Karma issues) (Krinkle)

22 March 2016

  • curprev 17:4017:40, 22 March 2016imported>Labslogbot 50,825 bytes +83 deploying https://gerrit.wikimedia.org/r/278926 (legoktm)

21 March 2016

  • curprev 21:5521:55, 21 March 2016imported>Labslogbot 50,742 bytes +1,183 zuul: almost all MediaWiki extensions migrated to run the npm job on Nodepool (with Node.js 4.3) T119143 . All tested. Will monitor the build results that ran overnight tomorrow (hashar)

19 March 2016

  • curprev 13:0413:04, 19 March 2016imported>Labslogbot 49,559 bytes +114 Jenkins: added ldap-labs-codfw.wikimedia.org as a fallback LDAP server T130446 (hashar)

18 March 2016

17 March 2016

  • curprev 21:1021:10, 17 March 2016imported>Labslogbot 49,371 bytes +318 updating scap on deployment-tin to test D133 (thcipriani)

16 March 2016

  • curprev 14:2814:28, 16 March 2016imported>Labslogbot 49,053 bytes +143 Updated jobs having the package manager cache system (castor) via https://gerrit.wikimedia.org/r/#/c/277774/ (hashar)

15 March 2016

  • curprev 15:1715:17, 15 March 2016imported>Labslogbot 48,910 bytes +542 added wikidata.beta.wmflabs.org in https://wikitech.wikimedia.org/wiki/Special:NovaAddress to deployment-cache-text04.deployment-prep.eqiad.wmflabs (jzerebecki)

14 March 2016

  • curprev 22:1822:18, 14 March 2016imported>Labslogbot 48,368 bytes +398 new jobs weren't processing in Zuul, lego fixed it and blamed Reedy (greg-g)

11 March 2016

  • curprev 22:4222:42, 11 March 2016imported>Labslogbot 47,970 bytes +235 deploying https://gerrit.wikimedia.org/r/276901 (legoktm)

10 March 2016

  • curprev 20:2220:22, 10 March 2016imported>Labslogbot 47,735 bytes +503 Nodepool Image ci-jessie-wikimedia-1457641052 in wmflabs-eqiad is ready (hashar)

9 March 2016

  • curprev 17:5617:56, 9 March 2016imported>Labslogbot 47,232 bytes +2,402 Cleaned up git clone state in deployment-tin.deployment-prep:/srv/mediawiki-staging/php-master and queued beta-code-update-eqiad to try again (T129371) (bd808)

7 March 2016

  • curprev 21:0321:03, 7 March 2016imported>Labslogbot 44,830 bytes +226 Nodepool upgraded to 0.1.1-wmf.4 , it no more waits 1 minute before deleted a used node | T118573 (hashar)

6 March 2016

  • curprev 10:2010:20, 6 March 2016imported>Labslogbot 44,604 bytes +83 deploying https://gerrit.wikimedia.org/r/274911 (legoktm)

4 March 2016

  • curprev 19:3119:31, 4 March 2016imported>Labslogbot 44,521 bytes +435 Nodepool Image ci-jessie-wikimedia-1457119603 in wmflabs-eqiad is ready - T128846 (hashar)
  • curprev 00:1100:11, 4 March 2016imported>Labslogbot 44,086 bytes +947 salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" (Krinkle)

2 March 2016

  • curprev 16:2216:22, 2 March 2016imported>Labslogbot 43,139 bytes +297 reloading zuul for 9398fa1..943f17b (jzerebecki)
  • curprev 01:1901:19, 2 March 2016imported>Labslogbot 42,842 bytes +1,454 force restarting zuul because the queue is very stuck https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Restart (thcipriani)

29 February 2016

  • curprev 16:2316:23, 29 February 2016imported>Labslogbot 41,388 bytes +447 salt -v '*slave*' cmd.run 'rm -fR /mnt/jenkins-workspace/workspace/mwext*jslint' T127362 (hashar)

28 February 2016

  • curprev 01:3001:30, 28 February 2016imported>Labslogbot 40,941 bytes +119 Rebooting integration-slave-precise-1012 – Might help T109704 (MySQL not running) (Krinkle)

26 February 2016

  • curprev 15:1415:14, 26 February 2016imported>Labslogbot 40,822 bytes +1,342 salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" T128191 (jzerebecki)

25 February 2016

  • curprev 22:3822:38, 25 February 2016imported>Labslogbot 39,480 bytes +890 beta: maybe deployment-jobunner01 is processing jobs a bit faster now. Seems like hhvm went wild (hashar)
  • curprev 00:3100:31, 25 February 2016imported>Labslogbot 38,590 bytes +1,255 running puppet on beta to update scap to latest packaged version: sudo salt -b '10%' -G 'deployment_target:scap/scap' cmd.run 'puppet agent -t' (thcipriani)

23 February 2016

22 February 2016

  • curprev 22:0622:06, 22 February 2016imported>Labslogbot 37,039 bytes +498 Restarted puppetmaster service on deployment-puppetmaster to "fix" error "invalid byte sequence in US-ASCII" (bd808)

20 February 2016

  • curprev 20:1920:19, 20 February 2016imported>Labslogbot 36,541 bytes +415 beta-code-update-eqiad job repeatedly stuck at "IRC notifier plugin" (Krinkle)

19 February 2016

  • curprev 12:0912:09, 19 February 2016imported>Labslogbot 36,126 bytes +289 killed https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/ been running for 13 hours. Blocked because slave went offline due to labs reboots yesterday (hashar)

18 February 2016

  • curprev 19:2019:20, 18 February 2016imported>Labslogbot 35,837 bytes +492 deploying https://gerrit.wikimedia.org/r/271583 and https://gerrit.wikimedia.org/r/271581, both no-ops (legoktm)

17 February 2016

  • curprev 07:3607:36, 17 February 2016imported>Labslogbot 35,345 bytes +65 deploying https://gerrit.wikimedia.org/r/271201 (legoktm)
  • curprev 01:0101:01, 17 February 2016imported>Labslogbot 35,280 bytes +1,635 attempting to turn off NFS on 52 instances on deployment-prep project (yuvipanda)

15 February 2016

14 February 2016

13 February 2016

12 February 2016

  • curprev 23:5423:54, 12 February 2016imported>Labslogbot 31,361 bytes +908 beta cluster broken since 20:30 UTC https://logstash-beta.wmflabs.org/#/dashboard/elasticsearch/fatalmonitor havent looked (hashar)

11 February 2016

10 February 2016

  • curprev 23:5423:54, 10 February 2016imported>Labslogbot 27,938 bytes +3,058 depooling Trusty slaves that only have 2GB of ram that is not enough. https://phabricator.wikimedia.org/T126545 (hashar_)
  • curprev 01:0301:03, 10 February 2016imported>Labslogbot 24,880 bytes +3,895 deploying https://gerrit.wikimedia.org/r/269576 (legoktm)

9 February 2016

  • curprev 00:3900:39, 9 February 2016imported>Labslogbot 20,985 bytes +2,174 gallium edited /usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/trigger/gerrit.py and modified: replication_timeout = 300 -> replication_timeout = 10 (hashar)

8 February 2016

  • curprev 00:5700:57, 8 February 2016imported>Labslogbot 18,811 bytes +87 deploying https://gerrit.wikimedia.org/r/268031 (legoktm[NE])

6 February 2016

5 February 2016

  • curprev 13:3013:30, 5 February 2016imported>Labslogbot 18,650 bytes +739 beta cleaning out /data/project/logs/archive was from pre logstash area. We no more log this way since May 2015 apparently (hashar)

4 February 2016

  • curprev 22:0822:08, 4 February 2016imported>Labslogbot 17,911 bytes +201 reloading zuul for bed7be1..f57b7e2 (jzerebecki)
  • curprev 00:5700:57, 4 February 2016imported>Labslogbot 17,710 bytes +2,091 Got deployment-bastion processing Jenkins jobs again via instructions left by my past self at https://phabricator.wikimedia.org/T72597#747925 (bd808)

2 February 2016

31 January 2016

  • curprev 13:3513:35, 31 January 2016imported>Labslogbot 13,530 bytes +453 Jenkins IRC bot started falling at Jan 30 01:04:00 2016 for whatever reason.... Should be fine now (hashar)

30 January 2016

  • curprev 12:4612:46, 30 January 2016imported>Labslogbot 13,077 bytes +110 integration-slave-jessie-1001 : fixed puppet.con server name and ran puppet (hashar)

29 January 2016

28 January 2016

26 January 2016

  • curprev 16:5116:51, 26 January 2016imported>Labslogbot 9,728 bytes +472 updated OCG to version 64050af0456a43344b32e3e93561a79207565eaf (cscott)

25 January 2016

  • curprev 18:5918:59, 25 January 2016imported>Labslogbot 9,256 bytes +525 started redis-server on deployment-redis01 by commenting out latency-monitor-threshold from the redis.conf (twentyafterfour)

24 January 2016

22 January 2016

  • curprev 23:5823:58, 22 January 2016imported>Labslogbot 8,583 bytes +1,274 removed skins from mwext-qunit workspace on trusty-1013 slave (legoktm)

21 January 2016

  • curprev 21:4121:41, 21 January 2016imported>Labslogbot 7,309 bytes +757 restored role::mail::mx on deployment-mx (hashar)
  • curprev 01:0001:00, 21 January 2016imported>Labslogbot 6,552 bytes +1,287 jenkins-deploy@deployment-bastion:/srv/mediawiki-staging/php-master/extensions$ git pull && git submodule update --init --recursive (jzerebecki)

19 January 2016

17 January 2016

16 January 2016

15 January 2016

14 January 2016

13 January 2016

  • curprev 21:0621:06, 13 January 2016imported>Labslogbot 214,628 bytes +353 beta cluster code is up to date again. Got delayed by roughly 4 hours. (hashar)

12 January 2016

  • curprev 23:3023:30, 12 January 2016imported>Labslogbot 214,275 bytes +277 deploying https://gerrit.wikimedia.org/r/263757 https://gerrit.wikimedia.org/r/263756 (legoktm)

11 January 2016

  • curprev 22:2522:25, 11 January 2016imported>Labslogbot 213,998 bytes +501 Deleting old references on Zuul-merger for mediawiki/core : <tt>/usr/share/python/zuul/bin/python /home/hashar/zuul-clear-refs.py --until 15 /srv/ssd/zuul/git/mediawiki/core </tt> (hashar)

7 January 2016

  • curprev 23:1623:16, 7 January 2016imported>Labslogbot 213,497 bytes +290 deleted /mnt/jenkins-workspace/workspace/mediawiki-extensions-qunit/src/extensions/PdfHandler/.git/refs/heads/wmf/1.26wmf16.lock on slave 1013 (legoktm)
  • curprev 01:2501:25, 7 January 2016imported>Labslogbot 213,207 bytes +835 reloading zuul for b0a5335..c16368a (jzerebecki)

5 January 2016

  • curprev 16:5416:54, 5 January 2016imported>Labslogbot 212,372 bytes +322 Removed elastic search from CI slaves https://phabricator.wikimedia.org/T89083 https://gerrit.wikimedia.org/r/#/c/259301/ (hashar_)

4 January 2016

2 January 2016

1 January 2016

  • curprev 22:1722:17, 1 January 2016imported>Labslogbot 211,782 bytes +121 No nodepool ci-jessie-* hosts seen in Jenkins interface and rake-jessie jobs backing up (bd808)

30 December 2015

  • curprev 00:1300:13, 30 December 2015imported>Labslogbot 211,661 bytes +779 rake-jessie jobs running again which will hopefully clear the large zuull backlog (bd808)

26 December 2015

  • curprev 21:4121:41, 26 December 2015imported>Labslogbot 210,882 bytes +618 integration: getting rid of $wgHTTPProxy https://gerrit.wikimedia.org/r/261096 (no more needed) (hashar)

23 December 2015

  • curprev 23:3723:37, 23 December 2015imported>Labslogbot 210,264 bytes +192 Reloading Zuul to deploy I39b9f292e95363addf8983eec5d08a0af527a163 (marxarelli)

22 December 2015

  • curprev 23:3123:31, 22 December 2015imported>Labslogbot 210,072 bytes +496 (when npm jobs run) - sudo rm -rf /mnt/home/jenkins-deploy/.npm at integration-slave-trusty-1015 (due to cache corruption) (Krinkle)

21 December 2015

19 December 2015

  • curprev 22:1822:18, 19 December 2015imported>Labslogbot 209,035 bytes +412 sudo rm -rf /mnt/home/jenkins-deploy/tmpfs/jenkins* on integration-slave-precise-1014 (Krinkle)
  • curprev 01:2301:23, 19 December 2015imported>Labslogbot 208,623 bytes +1,774 Added Reedy as a projectadmin in the integration project (bd808)

17 December 2015

  • curprev 23:0723:07, 17 December 2015imported>Labslogbot 206,849 bytes +1,799 sudo salt --show-timeout '*slave*' cmd.run 'rm -fR /mnt/home/jenkins-deploy/tmpfs/jenkins-?/*' (legoktm)

16 December 2015

  • curprev 23:2323:23, 16 December 2015imported>Labslogbot 205,050 bytes +877 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/259616 (Krinkle)

15 December 2015

  • curprev 22:5722:57, 15 December 2015imported>Labslogbot 204,173 bytes +881 On scandium created /srv/ssd/zuul/git/wikimedia/fundraising/crm repo manually. Namespace conflict with wikimedia/fundraising/crm/civicrm.git which prevented zuul-merger to clone the arm repo (hashar)
  • curprev 00:5700:57, 15 December 2015imported>Labslogbot 203,292 bytes +613 marking integration-slave-trusty-1012 offline, strange zuul.cloner behavior. (thcipriani)

13 December 2015

  • curprev 23:3023:30, 13 December 2015imported>Labslogbot 202,679 bytes +171 Ran deployment-bastion:~bd808/cleanup-var-crap.sh and freed 846M on /var (bd808)

11 December 2015

  • curprev 22:1822:18, 11 December 2015imported>Labslogbot 202,508 bytes +909 Stopped zuul merger on gallium to have phabricator/extensions populated on scandium (namespacing issue). Restarted zuul-merger on gallium once done. (hashar)
  • curprev 01:1201:12, 11 December 2015imported>Labslogbot 201,599 bytes +945 deploying https://gerrit.wikimedia.org/r/258395 (legoktm)

10 December 2015

8 December 2015

  • curprev 20:3120:31, 8 December 2015imported>Labslogbot 199,584 bytes +1,662 beta cluster instances switching to new ldap configuration (hashar)

7 December 2015

  • curprev 16:1616:16, 7 December 2015imported>Labslogbot 197,922 bytes +790 Nodepool no more listens for Jenkins events over ZeroMQ. No TCP connection established on port 8888 (hashar)

4 December 2015

3 December 2015

  • curprev 23:5323:53, 3 December 2015imported>Labslogbot 196,922 bytes +509 Reloading Zuul to deploy If60f720995dfc7859e53cf33043b5a21b1a4b085 (marxarelli)

2 December 2015

  • curprev 11:1611:16, 2 December 2015imported>Labslogbot 196,413 bytes +556 configure wmf-insecte to join #wikimedia-android-ci ( https://gerrit.wikimedia.org/r/#/c/254905/3/jjb/mobile.yaml,unified ) (hashar)

1 December 2015

  • curprev 16:0616:06, 1 December 2015imported>Labslogbot 195,857 bytes +584 split mediawiki core parser tests under Zend to their own job https://gerrit.wikimedia.org/r/#/c/256006/ (hashar)

29 November 2015

  • curprev 23:2323:23, 29 November 2015imported>Labslogbot 195,273 bytes +436 updated cherry-pick of https://gerrit.wikimedia.org/r/#/c/255916/ to PS2 (bd808)

27 November 2015

  • curprev 19:2519:25, 27 November 2015imported>Labslogbot 194,837 bytes +235 Trebuchet returners not reporting to redis from sca0[12] (bd808)

26 November 2015

  • curprev 22:3522:35, 26 November 2015imported>Labslogbot 194,602 bytes +553 Upgraded Zuul on labs Precise hosts to zuul_2.1.0-60-g1cc37f7-wmf4precise1 ( https://phabricator.wikimedia.org/T119741 ) (hashar)

25 November 2015

  • curprev 19:5319:53, 25 November 2015imported>Labslogbot 194,049 bytes +173 ran mwscript sql.php --wiki=enwiki --wikidb=wikishared /srv/mediawiki-staging/php-master/extensions/Echo/db_patches/echo_unread_wikis.sql (legoktm)

24 November 2015

  • curprev 21:1621:16, 24 November 2015imported>Labslogbot 193,876 bytes +378 bumping our JJB fork to 4eb33b2. Pinning jenkins==0.4.8 with https://gerrit.wikimedia.org/r/255182 (hashar)

23 November 2015

  • curprev 22:3222:32, 23 November 2015imported>Labslogbot 193,498 bytes +1,069 Updating parsoidsvc-source-npm-* jobs https://gerrit.wikimedia.org/r/254946 (hashar)

22 November 2015

  • curprev 21:1521:15, 22 November 2015imported>Labslogbot 192,429 bytes +110 Zuul queue stuck as of 3 hours ago. Jenkins unresponsive or 503 over HTTP. (Krinkle)

20 November 2015

  • curprev 21:5921:59, 20 November 2015imported>Labslogbot 192,319 bytes +266 Removed shadow l10nupdate user from /etc/passwd & /etc/shadow on mira.deployment-prep.eqiad.wmflabs (bd808)

13 November 2015

  • curprev 21:1121:11, 13 November 2015imported>Labslogbot 192,053 bytes +992 zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/tools/releng --change 200240,1 (hashar)

12 November 2015

  • curprev 22:0322:03, 12 November 2015imported>Labslogbot 191,061 bytes +94 rm -rf mediawiki-extensions-qunit workspace on trusty-1012 (legoktm)

11 November 2015

  • curprev 17:5817:58, 11 November 2015imported>Labslogbot 190,967 bytes +56 reloading zuul for e81ee95..acbd851 (jzerebecki)
  • curprev 00:0200:02, 11 November 2015imported>Labslogbot 190,911 bytes +500 deployment-puppetmaster:/var/lib/git/operations/puppet was left mid-rebase conflicts with portals apache config. Fixed conflicts added the file and continued the rebase. (thcipriani)

9 November 2015

  • curprev 23:1923:19, 9 November 2015imported>Labslogbot 190,411 bytes +630 put https://integration.wikimedia.org/ci/computer/integration-slave-trusty-1015/ back on (hashar)

7 November 2015

6 November 2015

5 November 2015

4 November 2015

  • curprev 23:1823:18, 4 November 2015imported>Labslogbot 188,479 bytes +820 Running `jenkins-jobs update config/ 'mw-tools-scap-tox-doc-publish'` to test Ib4753ad493115682d902cf15136199fd2083b8e5 (marxarelli)

3 November 2015

  • curprev 22:4822:48, 3 November 2015imported>Labslogbot 187,659 bytes +690 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/250847 (thcipriani)

2 November 2015

  • curprev 20:4820:48, 2 November 2015imported>Labslogbot 186,969 bytes +1,694 Upgraded Zuul on Trusty slaves to /root/zuul_2.1.0-60-g1cc37f7-wmf1trusty1_amd64.deb (hashar)

1 November 2015

31 October 2015

  • curprev 21:3221:32, 31 October 2015imported>Labslogbot 185,192 bytes +107 Ran `sudo rm /var/log/account/pacct.?` on deployment-bastion; freed ~425M (bd808)

30 October 2015

  • curprev 18:2018:20, 30 October 2015imported>Labslogbot 185,085 bytes +401 deployment-db2 replication recovered after slave stop/reset/set master position/start (marxarelli)

29 October 2015

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)