You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Release Engineering/SAL: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

17 October 2017

  • curprev 22:0822:08, 17 October 2017imported>Stashbot 245,375 bytes +729 addshore: replaced integration-slave-docker-c2-m4-d40-1005 with integration-slave-docker-1005 T178409

16 October 2017

14 October 2017

11 October 2017

  • curprev 19:5919:59, 11 October 2017imported>Stashbot 243,320 bytes +420 hashar: deployment-prep: deploying jobrunner to catchup with changes.

10 October 2017

  • curprev 17:5117:51, 10 October 2017imported>Stashbot 242,900 bytes +205 Amir1: add "Ladsgroup" to oversight members in enwiki in beta cluster to test T177705

9 October 2017

6 October 2017

  • curprev 13:2213:22, 6 October 2017imported>Stashbot 242,495 bytes +393 hashar: Jenkins: adding Maven-3.0.5 to the tool configuration https://integration.wikimedia.org/ci/configureTools/

5 October 2017

  • curprev 19:1519:15, 5 October 2017imported>Stashbot 242,102 bytes +728 hasharAway: rebooting integration-slave-docker-1002 to catch with kernel upgrade and pooling it back in Jenkins - T177039

4 October 2017

  • curprev 13:1913:19, 4 October 2017imported>Stashbot 241,374 bytes +92 andrewbogott: migrating 'deployment-kafka-jumbo-1' to labvirt1017

3 October 2017

  • curprev 22:3822:38, 3 October 2017imported>Stashbot 241,282 bytes +491 thcipriani: git stash /srv/mediawiki-staging/php-master/extensions/Echo to fix beta-code-update-eqiad

2 October 2017

  • curprev 15:5215:52, 2 October 2017imported>Stashbot 240,791 bytes +831 zeljkof: Reloading Zuul to deploy 895f33da2a8daeb1fa2634bcd4eddfbbbcd78819

30 September 2017

  • curprev 14:2814:28, 30 September 2017imported>Stashbot 239,960 bytes +112 zeljkof: Reloading Zuul to deploy c08a3ad67b5bc1cbdb900d6b4138034b4be83c0f

29 September 2017

27 September 2017

  • curprev 21:2521:25, 27 September 2017imported>Stashbot 236,998 bytes +1,825 hashar: salt is being replaced by cumin instances being deployment-cumin and integration-cumin . Check this out: https://wikitech.wikimedia.org/wiki/Cumin !

26 September 2017

  • curprev 16:1516:15, 26 September 2017imported>Stashbot 235,173 bytes +876 addshore: docker push docker.io/wmfreleng/zuul-cloner:v2017.09.26.16.09 & latest (from PS11 of https://gerrit.wikimedia.org/r/379479)

25 September 2017

24 September 2017

  • curprev 10:4910:49, 24 September 2017imported>Stashbot 233,787 bytes +545 addshore: addshore@integration-saltmaster:~$ sudo salt -v '*slave-docker*' cmd.run "sudo docker images --no-trunc --format '{{.ID}} {{.CreatedSince}}' | grep ' months' | awk '{ print $1 }' | xargs --no-run-if-empty docke r rmi"

22 September 2017

  • curprev 21:5521:55, 22 September 2017imported>Stashbot 233,242 bytes +1,077 tabbycat: Granted Greg G. 'staff' global rights on the beta cluster per request

21 September 2017

  • curprev 10:2310:23, 21 September 2017imported>Stashbot 232,165 bytes +183 elukey: removed 6fdf6ee653 from deployment-prep's puppet master cherry picks (seemed an old version of https://gerrit.wikimedia.org/r/#/c/357985)

20 September 2017

  • curprev 15:4615:46, 20 September 2017imported>Stashbot 231,982 bytes +2,840 addshore: reloading zuul for https://gerrit.wikimedia.org/r/#/c/379250/

12 September 2017

  • curprev 20:3520:35, 12 September 2017imported>Stashbot 229,142 bytes +962 hashar: pooling integration-slave-jessie-1003 and integration-slave-jessie-1004

11 September 2017

10 September 2017

  • curprev 01:4401:44, 10 September 2017imported>Stashbot 227,970 bytes +192 bd808: nodepool running steadily again, but has been heavily throttled to hopefully prevent another weekend thundering herd of doom failure for the OpenStack backend

9 September 2017

  • curprev 22:1522:15, 9 September 2017imported>Stashbot 227,778 bytes +557 bd808: `sudo journalctl -u nodepool --since today --no-pager` shows many LaunchStatusException failures.

6 September 2017

  • curprev 21:3221:32, 6 September 2017imported>Stashbot 227,221 bytes +301 bearND: Update mobileapps to 2cb6281 (T168848 T169277 T169274 T162179 T164033 T167921 T174698 T168848 T174808)

5 September 2017

  • curprev 23:0323:03, 5 September 2017imported>Stashbot 226,920 bytes +204 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/376034

4 September 2017

  • curprev 15:5915:59, 4 September 2017imported>Stashbot 226,716 bytes +316 zeljkof: Reloading Zuul to deploy ca1c6ec05122e6f50a5a52fecbbd2bd8e882ad87
  • curprev 00:4200:42, 4 September 2017imported>Stashbot 226,400 bytes +115 legoktm: legoktm@contint1001:/srv/zuul/git/mediawiki/libs$ sudo -u zuul rm -rf XMPReader

2 September 2017

  • curprev 08:3208:32, 2 September 2017imported>Stashbot 226,285 bytes +102 legoktm: rm -rf /var/logs/kafka on deployment-kafka01 to free up disk space

31 August 2017

  • curprev 23:3223:32, 31 August 2017imported>Stashbot 226,183 bytes +756 Krenair: fixed deployment-imagescaler0[12] puppet by installing a package and file manually, some puppetisation still needed - https://phabricator.wikimedia.org/T174746

30 August 2017

  • curprev 12:4912:49, 30 August 2017imported>Stashbot 225,427 bytes +121 hashar: gerrit: marked wikimedia/communications/WMBlog as read-only - T172372

29 August 2017

  • curprev 15:3915:39, 29 August 2017imported>Stashbot 225,306 bytes +366 hashar: Created integration-slave-jessie-php55 to try out a php5.5 package on Jessie - T161882

28 August 2017

  • curprev 14:5414:54, 28 August 2017imported>Stashbot 224,940 bytes +294 hashar: integration: rebase integration puppet master. Got a conflict due to r -> r_lang renaming ( https://gerrit.wikimedia.org/r/#/c/363337/ )

25 August 2017

  • curprev 15:1115:11, 25 August 2017imported>Stashbot 224,646 bytes +112 zeljkof: Reloading Zuul to deploy b6704e249f0d9e83de5835908eaddff40f104e2f

24 August 2017

  • curprev 21:5521:55, 24 August 2017imported>Stashbot 224,534 bytes +422 mdholloway: disk was full on integration-slave-jessie-android; deleted ~8gb of old screenshots from /tmp to clear some space

22 August 2017

21 August 2017

18 August 2017

16 August 2017

  • curprev 23:4623:46, 16 August 2017imported>Stashbot 223,814 bytes +212 legoktm: reloading Zuul to deploy https://gerrit.wikimedia.org/r/372208 https://gerrit.wikimedia.org/r/371653 https://gerrit.wikimedia.org/r/371757 https://gerrit.wikimedia.org/r/371640

15 August 2017

  • curprev 17:5717:57, 15 August 2017imported>Stashbot 223,602 bytes +176 bearND: Update mobileapps to 33b80dd (T172829 T152441 T172021 T103362)

14 August 2017

  • curprev 09:4609:46, 14 August 2017imported>Stashbot 223,426 bytes +195 TabbyCat: maurelio@deployment-tin:/srv/mediawiki/dblists$ expanddblist flow-computed > /home/maurelio/flow-test.dblist (to test expandblist for a patch I am working on)

11 August 2017

  • curprev 20:2520:25, 11 August 2017imported>Stashbot 223,231 bytes +93 addshore: added mediawiki::maintenance::wikidata to deployment-tin

7 August 2017

6 August 2017

  • curprev 13:2813:28, 6 August 2017imported>Stashbot 223,063 bytes +134 TabbyCat: Ran mwscript extensions/WikimediaMaintenance/dumpInterwiki.php deploymentwiki on the beta cluster

4 August 2017

  • curprev 15:2115:21, 4 August 2017imported>Stashbot 222,929 bytes +107 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/370222/2

3 August 2017

  • curprev 12:0212:02, 3 August 2017imported>Stashbot 222,822 bytes +391 hashar: Added integration-slave-docker-1004 to the pool of jenkins slaves - T150502

2 August 2017

1 August 2017

  • curprev 15:4515:45, 1 August 2017imported>Stashbot 221,692 bytes +420 hashar: Image snapshot-ci-jessie-1501601670 in wmflabs-eqiad is ready && purging old instances T161861

28 July 2017

  • curprev 21:1121:11, 28 July 2017imported>Stashbot 221,272 bytes +160 MaxSem: Dropped table wikigrok_questions from beta enwiki
  • curprev 00:1700:17, 28 July 2017imported>Stashbot 221,112 bytes +244 Krinkle: Testing job insertion on beta cluster from deployment-tin triggers PHP Notice: Undefined index: uuid in EventBus/JobQueueEventBus.php:102, PHP Notice: Undefined index: sha1 in EventBus/JobQueueEventBus.php:99

26 July 2017

  • curprev 21:3521:35, 26 July 2017imported>Stashbot 220,868 bytes +780 Reedy: kill two long running update.php jobs on deployment-tin

25 July 2017

24 July 2017

21 July 2017

  • curprev 14:5514:55, 21 July 2017imported>Stashbot 218,220 bytes +558 hashar: Jenkins: upgraded Android Emulator plugin with https://gerrit.wikimedia.org/r/#/c/366253/ && https://gerrit.wikimedia.org/r/#/c/366484/ - T150623

20 July 2017

  • curprev 16:4216:42, 20 July 2017imported>Stashbot 217,662 bytes +1,886 hashar: How to fix ssh access on beta cluster instances: https://phabricator.wikimedia.org/T171174#3456966

19 July 2017

  • curprev 14:4314:43, 19 July 2017imported>Stashbot 215,776 bytes +448 hashar: Jenkins: uploaded a patched android-emulator plugin for T150623 and restarting Jenkins

18 July 2017

  • curprev 16:2616:26, 18 July 2017imported>Stashbot 215,328 bytes +414 halfak: manually restarted uwsgi-ores and celery-ores-worker on deployment-sca03

17 July 2017

  • curprev 18:2018:20, 17 July 2017imported>Stashbot 214,914 bytes +105 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/365198

14 July 2017

13 July 2017

12 July 2017

9 July 2017

  • curprev 01:1501:15, 9 July 2017imported>Stashbot 214,107 bytes +188 Amir1: ladsgroup@deployment-tin:~$ mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=enwiki (T170026, T165716)

7 July 2017

  • curprev 14:5414:54, 7 July 2017imported>Stashbot 213,919 bytes +559 hashar: deployment-prep: change webproxy http://recommendation-api-beta.wmflabs.org/ to deployment-sca02 (has the proper security rule) - T148129

6 July 2017

  • curprev 17:2817:28, 6 July 2017imported>Stashbot 213,360 bytes +759 thcipriani: committed changes to modules/kafkatee on deployment-puppetmaster02 since having them uncommitted broke git-sync-upstream

4 July 2017

  • curprev 14:1014:10, 4 July 2017imported>Stashbot 212,601 bytes +402 hashar: manually upgraded apache2 on deployment-puppetmaster02 see T159254

3 July 2017

  • curprev 11:3411:34, 3 July 2017imported>Stashbot 212,199 bytes +143 hashar: jenkins: refreshing all jobs and updating the castor-save bit ( https://gerrit.wikimedia.org/r/#/c/361843/ )

30 June 2017

  • curprev 08:1608:16, 30 June 2017imported>Stashbot 212,056 bytes +165 hashar: Gerrit: changing repos to read-only: analytics/kraken analytics/kraken/deploy analytics/vagrant/kraken - T169303

29 June 2017

  • curprev 23:1723:17, 29 June 2017imported>Stashbot 211,891 bytes +83 legoktm: deploying https://gerrit.wikimedia.org/r/362314

28 June 2017

  • curprev 15:5515:55, 28 June 2017imported>Stashbot 211,808 bytes +866 hashar: beta: git gc mediawiki repos in /srv/mediawiki-staging

27 June 2017

26 June 2017

  • curprev 22:2422:24, 26 June 2017imported>Stashbot 210,291 bytes +777 halfak: deploying ores-prod-deploy:82dfd56 to beta (note: T168099)

24 June 2017

  • curprev 06:0206:02, 24 June 2017imported>Stashbot 209,514 bytes +159 legoktm: deployment-flourine02 /srv partition is alerting on low disk space but once logs get automatically gzip'd it should be fine

23 June 2017

  • curprev 20:5920:59, 23 June 2017imported>Stashbot 209,355 bytes +191 hasharAway: deployment-db03 reinstall ldap-utils, libldap-2.4-2 2.4.44+dfsg-4~bpo8+1 > 2.4.41+dfsg-1+wmf1

22 June 2017

21 June 2017

20 June 2017

  • curprev 19:2519:25, 20 June 2017imported>Stashbot 205,046 bytes +142 hashar: Nodepool rate being bumped from 1 query per 6 seconds to 1 query per 5 seconds ( https://gerrit.wikimedia.org/r/#/c/358601/ )
  • curprev 01:2501:25, 20 June 2017imported>Stashbot 204,904 bytes +282 thcipriani|afk: deployment-tin stuck on post-merge queue for the past 13 hours, unstuck now

18 June 2017

  • curprev 19:2619:26, 18 June 2017imported>Stashbot 204,622 bytes +266 Reedy: Re-enabled beta-update-databases-eqiad as wikidatawiki takes < 10 minutes T168036 T167981

16 June 2017

  • curprev 23:4123:41, 16 June 2017imported>Stashbot 204,356 bytes +1,158 Reedy_: also deleting a lot of Property:P* pages on beta wikidatawiki T168106

15 June 2017

  • curprev 16:3416:34, 15 June 2017imported>Stashbot 203,198 bytes +150 RainbowSprinkles: deployment-prep: Disabled database updates for awhile, running it by hand
  • curprev 00:5200:52, 15 June 2017imported>Stashbot 203,048 bytes +295 thcipriani: deployment-tin jenkins agent borked for 4 hours, should be fixed now

13 June 2017

  • curprev 22:0522:05, 13 June 2017imported>Stashbot 202,753 bytes +1,800 hashar: Zuul resarted manually from a terminal on contint1001. It does not have any statsd configuration so we will miss metrics for a bit till it is restarted properly.

12 June 2017

  • curprev 14:2214:22, 12 June 2017imported>Stashbot 200,953 bytes +534 hashar: Image snapshot-ci-trusty-1497276913 in wmflabs-eqiad is ready

9 June 2017

  • curprev 20:3020:30, 9 June 2017imported>Stashbot 200,419 bytes +196 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/358092/1

7 June 2017

  • curprev 17:4917:49, 7 June 2017imported>Stashbot 200,223 bytes +616 elukey: forced /usr/local/bin/git-sync-upstream manually on puppetmaster02

6 June 2017

  • curprev 14:3414:34, 6 June 2017imported>Stashbot 199,607 bytes +587 hashar: deleting buildlog.integration.eqiad.wmflabs was mean to receive Jenkins logs in ElasticSearch. We are experimenting with relforge1001.eqiad.wmnet now - T78705

5 June 2017

  • curprev 15:3815:38, 5 June 2017imported>Stashbot 199,020 bytes +125 elukey: manually hacking deployment-jobrunner02.deployment-prep.eqiad.wmflabs to test a new config

2 June 2017

  • curprev 19:5119:51, 2 June 2017imported>Stashbot 198,895 bytes +220 hashar: integration: granted ebernhardson sudo

1 June 2017

31 May 2017

  • curprev 20:2120:21, 31 May 2017imported>Stashbot 198,024 bytes +413 hashar: Jenkins: upgrading git-client-plugin 2.4.5..2.4.6 T166557

30 May 2017

  • curprev 22:0822:08, 30 May 2017imported>Stashbot 197,611 bytes +157 hasharAway: Changed integration/config.git submit type from "Fast forward only" to "Rebase if Necessary" T131008

29 May 2017

26 May 2017

  • curprev 09:2009:20, 26 May 2017imported>Stashbot 196,844 bytes +139 elukey: installing hhvm_3.18.2+dfsg-1+wmf4+exp1_amd64.deb on jobrunner02
  • curprev 01:2801:28, 26 May 2017imported>Stashbot 196,705 bytes +551 bearND: Update mobileapps to db6493c

24 May 2017

  • curprev 16:0416:04, 24 May 2017imported>Stashbot 196,154 bytes +518 hashar: rebooting integration-slave-trusty-1003 to catch up with kernel upgrade

23 May 2017

22 May 2017

  • curprev 12:1112:11, 22 May 2017imported>Stashbot 195,305 bytes +226 greg-g: ran git prune and rm'd the gc.log file

21 May 2017

20 May 2017

19 May 2017

  • curprev 19:0519:05, 19 May 2017imported>Stashbot 194,663 bytes +739 mutante: fixing role class config on deployment-phab* (remove role::phabricator::main, add role::phabricator_server in context prefix "deployment-phab. remove again from instance level for phab-01

15 May 2017

  • curprev 10:4610:46, 15 May 2017imported>Stashbot 193,924 bytes +163 addshore: enabled beta-code-update-eqiad for some testing

13 May 2017

  • curprev 20:3120:31, 13 May 2017imported>Stashbot 193,761 bytes +158 bd808: Deleted stuck mediawiki-core-doxygen-publish job. Jenkins had it marked for a particular nodepool instance that was offline.

12 May 2017

  • curprev 13:1213:12, 12 May 2017imported>Stashbot 193,603 bytes +136 hashar: Trying to refresh Nodepool Jessie image. Should get HHVM pinned to 'experimental' component => 3.12.x

11 May 2017

  • curprev 20:4320:43, 11 May 2017imported>Stashbot 193,467 bytes +303 hashar: nodepool: delete today jessie image snapshot. It comes with HHVM 3.18 which segfault with MediaWiki/PHPUnit. Rolled back to snapshot-ci-jessie-1494425642 from 30 hours ago. T165074

10 May 2017

9 May 2017

  • curprev 12:4412:44, 9 May 2017imported>Stashbot 191,498 bytes +248 hashar: deployment-ircd upgrading puppet 3.7.2 => 3.8.5

8 May 2017

  • curprev 21:4221:42, 8 May 2017imported>Stashbot 191,250 bytes +88 thcipriani: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/351131/
  • curprev 00:5700:57, 8 May 2017imported>Stashbot 191,162 bytes +99 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/351130

6 May 2017

  • curprev 01:1601:16, 6 May 2017imported>Stashbot 191,063 bytes +634 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/352154

4 May 2017

  • curprev 10:4710:47, 4 May 2017imported>Stashbot 190,429 bytes +1,197 hashar: puppet ca destroy deployment-zookeeper01.eqiad.wmflabs

3 May 2017

  • curprev 17:3917:39, 3 May 2017imported>Stashbot 189,232 bytes +564 mdholloway: (this concerns integration-slave-jessie-android)

2 May 2017

  • curprev 21:1421:14, 2 May 2017imported>Stashbot 188,668 bytes +897 hashar: Manually cancelled a few mediawiki-core-jsduck-publish and mediawiki-core-doxygen-publish job in Jenkins build queue. They seems to deadlock Jenkins somehow :(

1 May 2017

  • curprev 20:3920:39, 1 May 2017imported>Stashbot 187,771 bytes +967 hashar: Updated REL1_29 branch of ImportArticles / OAuth / Quiz and Wikispeech so they get phpcs ( https://gerrit.wikimedia.org/r/#/c/350984/ )

25 April 2017

22 April 2017

  • curprev 20:1720:17, 22 April 2017imported>Stashbot 186,633 bytes +102 hashar: Added FlorianSW to Github organization "wikimedia" (no team though)

21 April 2017

  • curprev 12:2512:25, 21 April 2017imported>Stashbot 186,531 bytes +356 hashar: T104048 zuul enqueue --trigger gerrit --pipeline postmerge --project AhoCorasick --change 345433,1

20 April 2017

  • curprev 17:0917:09, 20 April 2017imported>Stashbot 186,175 bytes +272 elukey: reverted hack on deployment-tin (apparently no effects on the jobrunner)

19 April 2017

  • curprev 16:3416:34, 19 April 2017imported>Stashbot 185,903 bytes +339 hashar: deleted nodepool alien ci-jessie-wikimedia-613597

18 April 2017

  • curprev 14:2914:29, 18 April 2017imported>Stashbot 185,564 bytes +411 hashar: unbreaking integration puppetmaster. Broke it when upgrading the puppet package :(

14 April 2017

  • curprev 12:2912:29, 14 April 2017imported>Stashbot 185,153 bytes +588 hashar: Delete integration-c1 instance (32GB RAM) on labvirt1004. It was used as a workaround for T161006
  • curprev 00:5900:59, 14 April 2017imported>Stashbot 184,565 bytes +579 Amir1: three cherry-picks failed to merge, skipped them 93dad5bec8e937ef93bdd63046b0bbbf14ad9722 92c7d0b002a02ff46a00e79d6d89fe83d5f65c17 21d60a478ffb21160049679eea235bdb1010a489

12 April 2017

  • curprev 15:1415:14, 12 April 2017imported>Stashbot 183,986 bytes +876 hashar: rm -fR /mnt/home/jenkins-deploy/.android/build-cache/* # T162635

11 April 2017

  • curprev 15:5915:59, 11 April 2017imported>Stashbot 183,110 bytes +1,626 hashar: integration-config-tox-jessie job is broken due to the JJB upgrade

10 April 2017

  • curprev 21:1721:17, 10 April 2017imported>Stashbot 181,484 bytes +517 hashar: marked a nodepool node online manually. The instance was up but Jenkins failed to reach it due to some SEVERE: I/O error in channel

6 April 2017

5 April 2017

  • curprev 13:3413:34, 5 April 2017imported>Stashbot 180,709 bytes +103 ema: testing possible fix for T162035 on deployment-ms-fe01

4 April 2017

  • curprev 21:2921:29, 4 April 2017imported>Stashbot 180,606 bytes +544 hashar: contint1001 : rm -fR /srv/zuul/git/mediawiki/services/graphoid/deploy due to T157818

3 April 2017

1 April 2017

  • curprev 09:4809:48, 1 April 2017imported>Stashbot 179,528 bytes +96 Sagan: puppet on deployment-tin looks like it is not running properly

29 March 2017

  • curprev 23:5123:51, 29 March 2017imported>Stashbot 179,432 bytes +447 Krinkle: Free up space on integration-slave-jessie-1001 by removing old /srv/jenkins-workspace and /srv/pbuilder dirs

28 March 2017

  • curprev 19:5319:53, 28 March 2017imported>Stashbot 178,985 bytes +624 hashar: Populating package manager cache of oojs-ui-npm-run-jenkins-node-6-jessie by manually triggering a build with ZUUL_PIPELINE=postmerge T155483

27 March 2017

  • curprev 17:0217:02, 27 March 2017imported>Stashbot 178,361 bytes +603 ebernhardson: cherry pick https://gerrit.wikimedia.org/r/344964 to puppetmaster to test upgrade to logstash 5.x

25 March 2017

24 March 2017

23 March 2017

22 March 2017

  • curprev 09:4509:45, 22 March 2017imported>Stashbot 177,103 bytes +350 hashar: beta: purging all Linux kernel from Swift instances

21 March 2017

  • curprev 16:4716:47, 21 March 2017imported>Stashbot 176,753 bytes +644 halfak: halfak@deployment-ores-redis:~$ redis-cli -h deployment-ores-redis.deployment-prep.eqiad.wmflabs -p 6380 -a areallysecretpassword flushall (T160762)

20 March 2017

  • curprev 20:5120:51, 20 March 2017imported>Stashbot 176,109 bytes +356 andrewbogott: migrating deployment-urldownloader to labvirt1013

17 March 2017

  • curprev 14:0814:08, 17 March 2017imported>Stashbot 175,753 bytes +147 hashar: salt -v '*precise*' cmd.run 'puppet agent --disable "Pending shutdown on March 20th - T158652"'

16 March 2017

  • curprev 21:4821:48, 16 March 2017imported>Stashbot 175,606 bytes +229 thcipriani: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/343113/

15 March 2017

14 March 2017

  • curprev 19:2319:23, 14 March 2017imported>Stashbot 173,290 bytes +288 thcipriani: removed alien nodepool instance via: openstack server delete ci-jessie-wikimedia-566503

13 March 2017

  • curprev 13:1913:19, 13 March 2017imported>Stashbot 173,002 bytes +551 hashar: Depooled Precise instances from Jenkins T158652 leaving the instances up for now.

10 March 2017

9 March 2017

  • curprev 16:2016:20, 9 March 2017imported>Stashbot 172,341 bytes +190 gehel: upgrading elasticsearch on deployment-prep to v5.1.2

8 March 2017

  • curprev 22:4522:45, 8 March 2017imported>Stashbot 172,151 bytes +121 Reedy: https://gerrit.wikimedia.org/r/#/c/341916/ cherry picked onto deployment-puppetmaster02

7 March 2017

  • curprev 22:3922:39, 7 March 2017imported>Stashbot 172,030 bytes +393 hashar: upgrading jenkins02.ci-staging to jenkins 2.x

6 March 2017

  • curprev 19:0419:04, 6 March 2017imported>Stashbot 171,637 bytes +175 addshore: mwscript sql.php --wiki=aawiki "CREATE DATABASE cognate_wiktionary"

2 March 2017

  • curprev 20:4720:47, 2 March 2017imported>Stashbot 171,462 bytes +1,230 hashar: deployment-prep: restarted apache/puppet master. Maybe that will fix ssh_known_hosts being emptied from time to time T159332

1 March 2017

  • curprev 19:0919:09, 1 March 2017imported>Stashbot 170,232 bytes +640 addshore: "mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki he wiktionary hewiktionary he.wiktionary.beta.wmflabs.org" T158628
  • curprev 01:0301:03, 1 March 2017imported>Stashbot 169,592 bytes +384 Reedy: beta-scap-eqiad giving Host key verification failed

27 February 2017

25 February 2017

  • curprev 03:5003:50, 25 February 2017imported>Stashbot 169,134 bytes +124 MaxSem: deployment-prep Deleted January logs from deployment-fluorine02, was running out of space

24 February 2017

  • curprev 13:5613:56, 24 February 2017imported>Stashbot 169,010 bytes +454 hashar: Log refresh Nodepool instances to deploy slave script update to be able to merge mediawiki/composer.json into vendor/composer.json 6527f49..a7728a5 https://gerrit.wikimedia.org/r/#/c/339202/ T158674

23 February 2017

  • curprev 18:3518:35, 23 February 2017imported>Stashbot 168,556 bytes +284 greg-g: 18:29 < chasemp> !log labnodepool1001:~# service nodepool restart

22 February 2017

  • curprev 20:5920:59, 22 February 2017imported>Stashbot 168,272 bytes +257 hashar: Deleted jenkins job pplint-HEAD. Fully replaced by rake / puppet-syntax gem - T154894

20 February 2017

  • curprev 14:5314:53, 20 February 2017imported>Stashbot 168,015 bytes +227 hashar: integration: applying role::ci::slave::saucelabs to saucelabs-01

17 February 2017

  • curprev 14:0814:08, 17 February 2017imported>Stashbot 167,788 bytes +160 hashar: integration: deleting "repository" instance. No time to figure out how to ship Sonatype Nexus to it. T147635

16 February 2017

15 February 2017

  • curprev 15:4715:47, 15 February 2017imported>Stashbot 167,089 bytes +267 hashar: Zuul reducing gate-and-submit minimum amount of changes to process from the wrong 12 down to 2. In case of repeating failures it would end up running jobs for only two jobs which would prevent cancelling jobs for up to 11 changes!

14 February 2017

  • curprev 14:3814:38, 14 February 2017imported>Stashbot 166,822 bytes +184 hashar: Updating castor-save publish job to properly capture composer cache on Jessie ( it is in ~/.composer/cache for some reason) T156359

13 February 2017

10 February 2017

9 February 2017

  • curprev 22:2222:22, 9 February 2017imported>Stashbot 165,668 bytes +369 greg-g: manually kicked off a bunch of selenium tests after tgr and Reedy fixed T157636

8 February 2017

7 February 2017

6 February 2017

4 February 2017

3 February 2017

  • curprev 11:0911:09, 3 February 2017imported>Stashbot 164,674 bytes +425 hashar: beta: removed old kernels from deployment-redis02 to free up disk space

2 February 2017

  • curprev 21:5621:56, 2 February 2017imported>Stashbot 164,249 bytes +680 hashar: integration-slave-jessie-1001 wiping /srv/pbuilder/base-trusty-amd64.cow it was not properly provisioned causing build to fail (eg lack of /etc/hosts) Running puppet to reprocvision it (poke T156651)
  • curprev 00:5200:52, 2 February 2017imported>Stashbot 163,569 bytes +881 tgr: added mhurd as member

31 January 2017

  • curprev 22:1222:12, 31 January 2017imported>Stashbot 162,688 bytes +328 thcipriani: started mysql on all integration precise instances via salt -- was stopped for some reason

26 January 2017

  • curprev 14:2514:25, 26 January 2017imported>Stashbot 162,360 bytes +307 hashar: Created Github repo for Gerrit replication https://github.com/wikimedia/mediawiki-libs-phpstorm-stubs T153252

24 January 2017

  • curprev 11:0411:04, 24 January 2017imported>Stashbot 162,053 bytes +171 hashar: Deleting integration-publisher (Precise) replaced by integration-publishing (Jessie). T156064 T143349

23 January 2017

21 January 2017

  • curprev 21:2021:20, 21 January 2017imported>Stashbot 161,313 bytes +417 hashar: integration: updating slave scripts for https://gerrit.wikimedia.org/r/#/c/333389/

20 January 2017

  • curprev 10:1410:14, 20 January 2017imported>Stashbot 160,896 bytes +485 hashar: puppet fails on "integration" labs instances due to an attempt to unmount the non existing NFS /home. Filled T155820

19 January 2017

  • curprev 22:1122:11, 19 January 2017imported>Stashbot 160,411 bytes +607 Krenair: added bunch of others to the same group per request. we should figure out how to make this process sane somehow

18 January 2017

  • curprev 10:4910:49, 18 January 2017imported>Stashbot 159,804 bytes +306 hashar: Disconnected/connected Jenkins Gearman client. The beta cluster builds had a deadlock.

17 January 2017

  • curprev 21:4721:47, 17 January 2017imported>Stashbot 159,498 bytes +657 urandom: deployment-prep restarting Cassandra on deployment-restbase02

16 January 2017

  • curprev 09:3309:33, 16 January 2017imported>Stashbot 158,841 bytes +226 hashar: integration nuked the Zuul merger path for SelectTag mw extension ( on scandium /srv/ssd/zuul/git/mediawiki/extensions/SelectTag ) Failed to merge https://gerrit.wikimedia.org/r/#/c/331974/

12 January 2017

  • curprev 00:3300:33, 12 January 2017imported>Stashbot 158,615 bytes +297 legoktm: deploying https://gerrit.wikimedia.org/r/331796 and https://gerrit.wikimedia.org/r/331795

10 January 2017

  • curprev 23:0723:07, 10 January 2017imported>Stashbot 158,318 bytes +98 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/331099

8 January 2017

  • curprev 05:2005:20, 8 January 2017imported>Stashbot 158,220 bytes +174 Krenair: deployment-stream: live hacked /usr/lib/python2.7/dist-packages/socketio/handler.py a bit (added apostrophes) to try to make rcstream work

7 January 2017

  • curprev 10:1710:17, 7 January 2017imported>Stashbot 158,046 bytes +131 Amir1: ladsgroup@deployment-tin:~$ mwscript updateCollation.php --wiki=fawiki (T139110)

6 January 2017

  • curprev 16:3116:31, 6 January 2017imported>Stashbot 157,915 bytes +275 hashar: Nodepool Image ci-jessie-wikimedia-1483719758 in wmflabs-eqiad is ready

5 January 2017

  • curprev 17:2017:20, 5 January 2017imported>Stashbot 157,640 bytes +194 hashar: Dropping puppet source from https://doc.wikimedia.org/puppetsource/ . contint1001: sudo rm -fR /srv/org/wikimedia/doc/puppetsource (T143233)

4 January 2017

  • curprev 21:3021:30, 4 January 2017imported>Stashbot 157,446 bytes +434 mutante: deployment-cache-text-04 - running acme-setup command to debug .. Creating CSR /etc/acme/csr/beta_wmflabs_org.pem

2 January 2017

  • curprev 11:2211:22, 2 January 2017imported>Stashbot 157,012 bytes +421 hashar: Nodepool Image ci-jessie-wikimedia-1483355768 in wmflabs-eqiad is ready

27 December 2016

26 December 2016

  • curprev 12:0912:09, 26 December 2016imported>Stashbot 156,484 bytes +122 hashar: beta: restarted varnish.service and varnish-frontend.service on deployment-cache-text04

24 December 2016

  • curprev 09:0209:02, 24 December 2016imported>Stashbot 156,362 bytes +99 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/329038

23 December 2016

22 December 2016

21 December 2016

  • curprev 05:5705:57, 21 December 2016imported>Stashbot 156,090 bytes +210 Krinkle: Jenkins "Collapsing Console Sections" for PHPUnit was broken since "-d zend.enable_gc=0" was added to phpunit.php invocation. Updated pattern in Jenkins system configuration.

19 December 2016

  • curprev 21:2121:21, 19 December 2016imported>Stashbot 155,880 bytes +616 andrewbogott: and also python-functools32_3.2.3.2-3~bpo8+1_all.deb

16 December 2016

15 December 2016

  • curprev 21:0021:00, 15 December 2016imported>Stashbot 154,105 bytes +471 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/324368

14 December 2016

  • curprev 12:3812:38, 14 December 2016imported>Stashbot 153,634 bytes +97 elukey: created deployment-copper on deployment-prep as temporary test

13 December 2016

  • curprev 22:5222:52, 13 December 2016imported>Stashbot 153,537 bytes +426 thcipriani: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/327119

9 December 2016

8 December 2016

  • curprev 23:3523:35, 8 December 2016imported>Stashbot 152,963 bytes +1,642 legoktm: deploying https://gerrit.wikimedia.org/r/326048 https://gerrit.wikimedia.org/r/326050

7 December 2016

  • curprev 15:0415:04, 7 December 2016imported>Stashbot 151,321 bytes +398 hashar: Image ci-trusty-wikimedia-1481122712 in wmflabs-eqiad is ready T117418

6 December 2016

  • curprev 21:2021:20, 6 December 2016imported>Stashbot 150,923 bytes +490 hashar: Image ci-jessie-wikimedia-1481058839 in wmflabs-eqiad is ready T113342

5 December 2016

  • curprev 20:4320:43, 5 December 2016imported>Stashbot 150,433 bytes +672 hashar: Image ci-jessie-wikimedia-1480969940 in wmflabs-eqiad is ready (include trendingedits::packages which explicitly define the installation of librdkafka-dev' )

4 December 2016

  • curprev 15:2515:25, 4 December 2016imported>Stashbot 149,761 bytes +160 Krenair: Found a git-sync-upstream cron on deployment-mx for some reason... commented for now, but wtf was this doing on a MX server?

3 December 2016

2 December 2016

  • curprev 14:4014:40, 2 December 2016imported>Stashbot 149,411 bytes +145 hashar: added Tobias Gritschacher to Gerrit "integration" group so he can +2 patches on integration/* repositories \O/

1 December 2016

  • curprev 18:2018:20, 1 December 2016imported>Stashbot 149,266 bytes +339 elukey: removing https://gerrit.wikimedia.org/r/#/c/305536 from the puppet master via rebase -i (no-op for beta)

30 November 2016

  • curprev 17:2217:22, 30 November 2016imported>Stashbot 148,927 bytes +513 gehel: restart of logstash on deployment-logstash2 - upgrade to Java 8 - T151325

29 November 2016

  • curprev 15:2315:23, 29 November 2016imported>Stashbot 148,414 bytes +305 hashar: Image ci-jessie-wikimedia-1480432368 in wmflabs-eqiad is ready

28 November 2016

  • curprev 09:4809:48, 28 November 2016imported>Stashbot 148,109 bytes +393 hashar: Image ci-trusty-wikimedia-1480326016 in wmflabs-eqiad is ready

26 November 2016

  • curprev 16:1516:15, 26 November 2016imported>Stashbot 147,716 bytes +617 Reedy: killed /srv/jenkins-workspace/workspace/mediawiki-core-*/src and /srv/jenkins-workspace/workspace/mwext-*/src from integration slaves to get rid of borked MW dirs

24 November 2016

  • curprev 20:5020:50, 24 November 2016imported>Stashbot 147,099 bytes +493 hashar: make contint1001 Jenkins slave to only builds jobs with a label matching the node https://integration.wikimedia.org/ci/computer/contint1001/configure T86659

23 November 2016

  • curprev 15:0415:04, 23 November 2016imported>Stashbot 146,606 bytes +229 Krenair: fixed puppet on deployment-cache-text04 by manually enabling experimental apt repo, see T150660

22 November 2016

21 November 2016

  • curprev 12:4612:46, 21 November 2016imported>Stashbot 146,017 bytes +145 hashar: beta: Cherry picked puppet fix for udp2log https://gerrit.wikimedia.org/r/#/c/322639/ T151169

19 November 2016

  • curprev 00:1000:10, 19 November 2016imported>Stashbot 145,872 bytes +237 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/322370

17 November 2016

16 November 2016

15 November 2016

  • curprev 16:0316:03, 15 November 2016imported>Stashbot 143,568 bytes +120 hasharAway: adding thcipriani to the labs "git" project maintained by paladox

14 November 2016

12 November 2016

11 November 2016

  • curprev 23:4823:48, 11 November 2016imported>Stashbot 143,268 bytes +376 bd808: Updated _template/logstash on deployment-logstash2 to include change from https://gerrit.wikimedia.org/r/#/c/320441/

10 November 2016

  • curprev 09:3309:33, 10 November 2016imported>Stashbot 142,892 bytes +185 hashar: Image ci-jessie-wikimedia-1478770026 in wmflabs-eqiad is ready

9 November 2016

  • curprev 20:2720:27, 9 November 2016imported>Stashbot 142,707 bytes +284 Krenair: removed default SSH access from production host 208.80.154.135, the old gallium IP

8 November 2016

  • curprev 10:2310:23, 8 November 2016imported>Stashbot 142,423 bytes +141 hashar: refreshing all jenkins jobs to clear out potential live hack I made but can't remember on which jobs I did

7 November 2016

  • curprev 14:0114:01, 7 November 2016imported>Stashbot 142,282 bytes +128 gilles: Pointing deployment-imagescaler01.eqiad.wmflabs' puppet to puppetmaster.thumbor.eqiad.wmflabs

4 November 2016

  • curprev 13:2013:20, 4 November 2016imported>Stashbot 142,154 bytes +880 hashar: gerrit: created mediawiki/extensions/PageViewInfo.git and renamed user group extension-WikimediaPageViewInfo to extension-PageViewInfo T148775

3 November 2016

2 November 2016

1 November 2016

  • curprev 22:2222:22, 1 November 2016imported>Stashbot 140,221 bytes +498 Krenair: started mysql on -db03 to hopefully pull us out of read-only mode

31 October 2016

27 October 2016

25 October 2016

  • curprev 19:2219:22, 25 October 2016imported>Labslogbot 138,602 bytes +283 Python PyPi mirror has some issue. Impacts all CI jobs relying on tox https://status.python.org/ (hasharAway)

24 October 2016

  • curprev 16:1916:19, 24 October 2016imported>Labslogbot 138,319 bytes +150 upgrading deployment-puppetmaster to puppet 3.8.5 packages (andrewbogott)

21 October 2016

20 October 2016

  • curprev 23:3723:37, 20 October 2016imported>Labslogbot 138,089 bytes +164 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/317083 (Krinkle)

14 October 2016

  • curprev 21:1321:13, 14 October 2016imported>Labslogbot 137,925 bytes +569 Ran START SLAVE to restart replication after columns created directly on replica were deleted. (matt_flaschen)

13 October 2016

  • curprev 21:2121:21, 13 October 2016imported>Labslogbot 137,356 bytes +1,438 Deleted CI slaves integration-slave-jessie-1004 integration-slave-jessie-1005 integration-slave-trusty-1013 integration-slave-trusty-1014 integration-slave-trusty-1017 integration-slave-trusty-1018 (hashar)

12 October 2016

  • curprev 13:3713:37, 12 October 2016imported>Labslogbot 135,918 bytes +289 upgraded memcached on deployment-memc04 to 1.4.28-1.1+wmf1 as part of a perf experiment (T129963) - rollback: wipe https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep/host/deployment-memc04, apt-get remove memcached on deployment-memc04, puppet r...

11 October 2016

  • curprev 21:3521:35, 11 October 2016imported>Labslogbot 135,629 bytes +550 Force pushed Zuul patchqueue 5628f95...fc6a118 HEAD -> patch-queue/debian/precise-wikimedia (hasharAway)

10 October 2016

  • curprev 15:4515:45, 10 October 2016imported>Labslogbot 135,079 bytes +132 deployment-prep deployment-elastic0[5-8]: reduce the number of replicas to 1 max for all indices (dcausse)

7 October 2016

  • curprev 20:1020:10, 7 October 2016imported>Labslogbot 134,947 bytes +213 Created repository.integration.eqiad.wmflabs to play/Test Sonatype Nexus (hashar)

6 October 2016

  • curprev 14:4514:45, 6 October 2016imported>Labslogbot 134,734 bytes +273 deployment-mira disarmed/rearmed keyholder in an attempt to clear a Shinken alarm (hashar)

5 October 2016

  • curprev 19:3319:33, 5 October 2016imported>Labslogbot 134,461 bytes +145 removing mediawiki::conftool from deployment-mediawiki04, deployment-mediawiki06, deployment-mediawiki05 (andrewbogott)

4 October 2016

  • curprev 19:4319:43, 4 October 2016imported>Labslogbot 134,316 bytes +540 removed contint::slave_scripts and associated files from deployment-sca01 and deployment-sca02 (andrewbogott)
  • curprev 01:1401:14, 4 October 2016imported>Labslogbot 133,776 bytes +890 New scap command line autocompletions are now installed on deployment-tin and deployment-mira refs T142880 (twentyafterfour)

1 October 2016

  • curprev 09:4109:41, 1 October 2016imported>Labslogbot 132,886 bytes +137 beta: shutdown deployment-db1 and deployment-db2 . Databases have been migrated to other hosts T138778 (hashar)

29 September 2016

  • curprev 15:4315:43, 29 September 2016imported>Labslogbot 132,749 bytes +641 logstash-beta: refreshed the field list via https://logstash-beta.wmflabs.org/app/kibana#/settings/indices/logstash (hashar)

28 September 2016

  • curprev 23:5623:56, 28 September 2016imported>Labslogbot 132,108 bytes +1,698 Deleted varnish cache files on deployment-cache-upload04 to free up space, disk full (MaxSem)

27 September 2016

  • curprev 20:1120:11, 27 September 2016imported>Labslogbot 130,410 bytes +195 Reloading Zuul to deploy 3c3289aa1a for T143938 and T146783 (hashar)

26 September 2016

24 September 2016

  • curprev 20:0820:08, 24 September 2016imported>Labslogbot 129,651 bytes +400 deployment-tin is shutdown. Replaced by Jessie deployment-tin02 (hashar)

23 September 2016

  • curprev 20:2120:21, 23 September 2016imported>Labslogbot 129,251 bytes +686 integration: salt -v '*trusty*' cmd.run 'service mysql start' (hashar)

22 September 2016

  • curprev 19:2919:29, 22 September 2016imported>Labslogbot 128,565 bytes +241 switching Jenkins slaves workspace from /mnt/jenkins-workspace to /srv/jenkins-workspace (actually the same dir/inode on the filesystem) (hasharAway)

21 September 2016

20 September 2016

  • curprev 21:4921:49, 20 September 2016imported>Labslogbot 126,058 bytes +2,278 Deleting deployment-mira02 /srv was too small. Replaced by deployment-mira (hashar)

19 September 2016

17 September 2016

16 September 2016

  • curprev 21:0321:03, 16 September 2016imported>Labslogbot 122,412 bytes +1,025 deployment-tin did a git gc on /srv/deployment/ores That freed up disk space and cleared an alarm on co master mira02 (hashar)

15 September 2016

  • curprev 22:2922:29, 15 September 2016imported>Labslogbot 121,387 bytes +1,969 sudo salt '*precise*' cmd.run 'service mysql start', all mysql's are down (legoktm)

14 September 2016

  • curprev 21:3721:37, 14 September 2016imported>Labslogbot 119,418 bytes +537 integration: setting "ulimit -c 2097152" on all slaves due to Zend PHP segfaulting T142158 (hashar)

13 September 2016

  • curprev 17:0217:02, 13 September 2016imported>Labslogbot 118,881 bytes +1,525 re-enabling beta cluster jenkins jobs following maintenance window (marxarelli)

12 September 2016

  • curprev 14:4114:41, 12 September 2016imported>Labslogbot 117,356 bytes +588 applied base::firewall, beta::deployaccess, mediawiki::conftool, role::mediawiki::appserver to deployment-mediawiki04.deployment-prep.eqiad.wmflabs (Debian jessie instance) - T144006 (elukey)

11 September 2016

9 September 2016

8 September 2016

  • curprev 21:3321:33, 8 September 2016imported>Labslogbot 116,286 bytes +1,462 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/309413 " Inject PHP_BIN=php5 for php53 jobs" (hashar)

7 September 2016

6 September 2016

5 September 2016

  • curprev 14:0814:08, 5 September 2016imported>Labslogbot 112,066 bytes +130 Refreshing Nodepool base images for Trusty and Jessie. Managed to build new ones after T143769 (hashar)

2 September 2016

1 September 2016

  • curprev 19:0019:00, 1 September 2016imported>Labslogbot 111,759 bytes +670 T130861: Restarting Cassandra on deployment-restbase0[1-2] (urandom)

31 August 2016

  • curprev 23:4023:40, 31 August 2016imported>Labslogbot 111,089 bytes +986 forced puppet run on deployment-salt02. Had not run automatically for 8 hours (bd808)

30 August 2016

  • curprev 23:3123:31, 30 August 2016imported>Labslogbot 110,103 bytes +2,327 cherry-picking https://gerrit.wikimedia.org/r/#/c/307656/ fixed puppet on the elasticsearch machines! (yuvipanda)

29 August 2016

  • curprev 07:5607:56, 29 August 2016imported>Labslogbot 107,776 bytes +226 hard rebooting integration-slave-trusty-1012 via horizon and restarting puppet manually (hashar)

27 August 2016

  • curprev 20:5120:51, 27 August 2016imported>Labslogbot 107,550 bytes +412 integration: tweak sudo policy for jenkins-deploy running cowbuilder: env_keep+=DEB_BUILD_OPTIONS (hashar)

26 August 2016

  • curprev 22:2922:29, 26 August 2016imported>Labslogbot 107,138 bytes +551 deploying https://gerrit.wikimedia.org/r/307025 (legoktm)
  • curprev 00:2800:28, 26 August 2016imported>Labslogbot 106,587 bytes +795 restarted puppetmaster service on deployment-puppetmaster (twentyafterfour)

24 August 2016

23 August 2016

  • curprev 18:2118:21, 23 August 2016imported>Labslogbot 105,304 bytes +409 deploying https://gerrit.wikimedia.org/r/306257 (legoktm)

22 August 2016

18 August 2016

  • curprev 22:0322:03, 18 August 2016imported>Labslogbot 104,823 bytes +334 deployment-fluorine02: Hack 'datasets:x:10003:997::/home/datasets:/bin/bash' into /etc/passwd for T117028 (bd808)

17 August 2016

  • curprev 22:2822:28, 17 August 2016imported>Labslogbot 104,489 bytes +524 deploying https://gerrit.wikimedia.org/r/305408 (legoktm)

16 August 2016

  • curprev 23:1023:10, 16 August 2016imported>Labslogbot 103,965 bytes +350 max_servers at 6, seeing 6 allocated instances, still seeing 403 already used 10 of 10 instances :(( (thcipriani)

15 August 2016

  • curprev 21:3021:30, 15 August 2016imported>Labslogbot 103,615 bytes +710 update scap on beta to 3.2.3-1 bugfix release (thcipriani)

13 August 2016

10 August 2016

  • curprev 23:5623:56, 10 August 2016imported>Labslogbot 102,533 bytes +347 deploying https://gerrit.wikimedia.org/r/304149 (legoktm)

9 August 2016

  • curprev 23:2123:21, 9 August 2016imported>Labslogbot 102,186 bytes +412 ladsgroup@deployment-sca03:~$ sudo service celery-ores-worker restart (Amir1)
  • curprev 00:5600:56, 9 August 2016imported>Labslogbot 101,774 bytes +341 deploying https://gerrit.wikimedia.org/r/303726 (legoktm)

7 August 2016

  • curprev 12:0112:01, 7 August 2016imported>Labslogbot 101,433 bytes +604 Nodepool: can't spawn instances due to: Forbidden: Quota exceeded for instances: Requested 1, but already used 10 of 10 instances (HTTP 403) (hashar)

6 August 2016

  • curprev 12:3112:31, 6 August 2016imported>Labslogbot 100,829 bytes +240 restarting uwsgi-ores and celery-ores-worker in deployment-sca03 (Amir1)

5 August 2016

  • curprev 17:5417:54, 5 August 2016imported>Labslogbot 100,589 bytes +251 Cherry-picked https://gerrit.wikimedia.org/r/#/c/299825/3 for testing (bd808)
  • curprev 00:4100:41, 5 August 2016imported>Labslogbot 100,338 bytes +421 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/303113 (Krinkle)

1 August 2016

  • curprev 20:2820:28, 1 August 2016imported>Labslogbot 99,917 bytes +170 restarting deployment-ms-be01, not responding to ssh, mw-fe01 requests timing out (thcipriani)

29 July 2016

  • curprev 23:2723:27, 29 July 2016imported>Labslogbot 99,747 bytes +842 Rebooting deployment-logstash2; Console showed hung task timeouts (P3606) (bd808)

28 July 2016

  • curprev 21:4621:46, 28 July 2016imported>Labslogbot 98,905 bytes +580 xintegration: change sudo policy for jenkins-deploy to help on T141538 : env_keep+=WORKSPACE (hashar_)

27 July 2016

  • curprev 16:1316:13, 27 July 2016imported>Labslogbot 98,325 bytes +878 salt -v '*slave-trusty*' cmd.run 'service mysql start' ( was missing on integration-slave-trusty-1011.integration.eqiad.wmflabs ) (hashar)

26 July 2016

  • curprev 20:1320:13, 26 July 2016imported>Labslogbot 97,447 bytes +795 Zuul deployed https://gerrit.wikimedia.org/r/301093 which adds 'debian-glue' job on all of operations/debs/ repos (hashar)
  • curprev 01:0801:08, 26 July 2016imported>Labslogbot 96,652 bytes +341 deployed ores a291da1 in sca03, ores-beta.wmflabs.org works as expected (Amir1)

23 July 2016

  • curprev 20:0620:06, 23 July 2016imported>Labslogbot 96,311 bytes +316 Cleanup jobrunner01 logs via -- sudo logrotate --force /etc/logrotate.d/mediawiki_jobrunner (bd808)

22 July 2016

  • curprev 20:2620:26, 22 July 2016imported>Labslogbot 95,995 bytes +1,322 T141114 upgraded jenkins-debian-glue from v0.13.0 to v0.17.0 on integration-slave-jessie-1001 and integration-slave-jessie-1002 (hashar)

21 July 2016

  • curprev 16:1216:12, 21 July 2016imported>Labslogbot 94,673 bytes +640 Updated our JJB fork cherry picking f74501e781f by madhuvishy. Was made to support the maven release plugin. Branch bump is 10f2bcd..6fcaf39 (hashar)
  • curprev 00:1300:13, 21 July 2016imported>Labslogbot 94,033 bytes +1,279 Cherry-picked https://gerrit.wikimedia.org/r/#/c/299825/ to deployment-puppetmaster so wdqs nginx log parsing can be tested (bd808)

18 July 2016

  • curprev 20:5620:56, 18 July 2016imported>Labslogbot 92,754 bytes +385 Deleted deployment-fluorine:/srv/mw-log/archive/*-201605* freed 30 GB (thcipriani)

16 July 2016

15 July 2016

  • curprev 08:3408:34, 15 July 2016imported>Labslogbot 92,317 bytes +132 Unpooling integration-slave-precise-1002 will use it as a zuul-server test instance temporarily (hashar)

14 July 2016

  • curprev 18:5418:54, 14 July 2016imported>Labslogbot 92,185 bytes +316 deployment-prep manually edited elasticsearch.yml on deployment-elastic05 and restarted to get it listening on eth0. Still looking into why puppet wrote out wrong config file (ebernhardson)

13 July 2016

  • curprev 16:0516:05, 13 July 2016imported>Labslogbot 91,869 bytes +512 Installing Cassandra 2.2.6-wmf1 on deployment-restbase0[1-2].deployment-prep.eqiad.wmflabs : T126629 (urandom)

12 July 2016

  • curprev 20:2920:29, 12 July 2016imported>Labslogbot 91,357 bytes +643 integration: force running unattended upgrade on all instances: salt --batch 4 -v '*' cmd.run 'unattended-upgrade' . That upgrades diamond and hhvm among others. imagemagick-common has a prompt though (hashar)

11 July 2016

  • curprev 14:2414:24, 11 July 2016imported>Labslogbot 90,714 bytes +579 Removing ZeroMQ config from the Jenkins jobs. It is now enabled globally. T139923 (hashar)

9 July 2016

  • curprev 18:5718:57, 9 July 2016imported>Labslogbot 90,135 bytes +223 deploying https://gerrit.wikimedia.org/r/297731 and https://gerrit.wikimedia.org/r/298142 (legoktm)

8 July 2016

  • curprev 16:0816:08, 8 July 2016imported>Labslogbot 89,912 bytes +817 scandium: git -C /srv/ssd/zuul/git/mediawiki/services/graphoid remote set-head origin --auto (hashar)

7 July 2016

  • curprev 21:4121:41, 7 July 2016imported>Labslogbot 89,095 bytes +283 Chowned php-master/vendor back to jenkins-deploy (MaxSem)
  • curprev 01:1401:14, 7 July 2016imported>Labslogbot 88,812 bytes +1,085 Restarted logstash on deployment-logstash2 (bd808)

5 July 2016

  • curprev 21:5421:54, 5 July 2016imported>Labslogbot 87,727 bytes +201 CI has drained the gate-and-submit queue (hasharAway)

4 July 2016

  • curprev 18:5818:58, 4 July 2016imported>Labslogbot 87,526 bytes +170 Upgrading arcanist on permanent CI slaves since xhpast was broken T137770 (hashar)

3 July 2016

  • curprev 13:1013:10, 3 July 2016imported>Labslogbot 87,356 bytes +238 phabricator Update phab-01 and phab-05 (phab-02) and phab-03 to fix a security bug in phabricator (Did the update last night but forgot to log it) (paladox)

2 July 2016

  • curprev 13:3813:38, 2 July 2016imported>Labslogbot 87,118 bytes +74 reloading zuul for 15127b2..7e6a2e2 (jzerebecki)

30 June 2016

  • curprev 10:3110:31, 30 June 2016imported>Labslogbot 87,044 bytes +492 Deleting integration-slave-trusty-1015 . Can not bring up mysql T138074 and the ssh slave connection would not hold anyway. Must be broken somehow (hashar)

28 June 2016

27 June 2016

  • curprev 22:3222:32, 27 June 2016imported>Labslogbot 86,456 bytes +680 deployment-prep deployed gerrit.wikimedia.org/r/296279 to puppetmaster to test kibana4 role (eberhardson)

24 June 2016

  • curprev 15:0415:04, 24 June 2016imported>Labslogbot 85,776 bytes +372 switch apps-android-wikimedia-* jobs to Jessie T138506 (hashar)

23 June 2016

  • curprev 13:5813:58, 23 June 2016imported>Labslogbot 85,404 bytes +391 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/295691 (Krinkle)

17 June 2016

  • curprev 10:3510:35, 17 June 2016imported>Labslogbot 85,013 bytes +181 offlined integration-slave-trusty-1015 T138074 (jzerebecki)

14 June 2016

  • curprev 14:2214:22, 14 June 2016imported>Labslogbot 84,832 bytes +275 T136971 on tin MediaWiki 1.28.0-wmf.6, from 1.28.0-wmf.6, successfully checked out. Applying security patches (hashar)

13 June 2016

  • curprev 21:1121:11, 13 June 2016imported>Labslogbot 84,557 bytes +898 https://integration.wikimedia.org/ci/computer/integration-slave-trusty-1015/ put offline. Jenkins cant ssh / pool it for some reason (hashar)

10 June 2016

  • curprev 15:4915:49, 10 June 2016imported>Labslogbot 83,659 bytes +779 reloading zuul for 8c048fb..272d1ec (jzerebecki)

9 June 2016

  • curprev 18:4918:49, 9 June 2016imported>Labslogbot 82,880 bytes +372 restarting nutcracker on deployment-mediawiki02 (hashar)

8 June 2016

7 June 2016

  • curprev 19:2819:28, 7 June 2016imported>Labslogbot 82,364 bytes +449 Nodepool has troubles spawning instances probably due to on going (?) labs maintenance (hashar)

6 June 2016

  • curprev 19:2619:26, 6 June 2016imported>Labslogbot 81,915 bytes +308 Regenerating Nodepool snapshots for Trusty and Jessie (hasharAway)

4 June 2016

  • curprev 00:0900:09, 4 June 2016imported>Labslogbot 81,607 bytes +562 krinkle@integration-slave-trusty-1017:~$ sudo rm -rf /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/Babel (T86730) (Krinkle)

2 June 2016

  • curprev 12:1012:10, 2 June 2016imported>Labslogbot 81,045 bytes +129 Upgraded Zuul upstream code being 66c8e52..30a433b package is 2.1.0-151-g30a433b-wmf1precise1 (hashar)

1 June 2016

  • curprev 17:4917:49, 1 June 2016imported>Labslogbot 80,916 bytes +888 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/292186 (Krinkle)

31 May 2016

  • curprev 20:2420:24, 31 May 2016imported>Labslogbot 80,028 bytes +101 Reloading zuul to pick up I58f878f3fd19dfa21a46a52464575cb06aacbb22 (bd808)

30 May 2016

  • curprev 18:3918:39, 30 May 2016imported>Labslogbot 79,927 bytes +624 Upgraded our Jenkins Job Builder fork to 1.5.0 + a couple of cherry picks: cd63874...10f2bcd (hashar)

28 May 2016

  • curprev 04:4304:43, 28 May 2016imported>Labslogbot 79,303 bytes +95 depooling integration-slave-trusty-1015 to profile phpunit runs (ori)

27 May 2016

26 May 2016

  • curprev 22:2322:23, 26 May 2016imported>Labslogbot 78,916 bytes +588 salt -v '*trusty*' cmd.run 'puppet agent --disable "Chromium needs to be v49. See T136188"' (hashar)

25 May 2016

  • curprev 22:3822:38, 25 May 2016imported>Labslogbot 78,328 bytes +1,025 running puppet agent manually on sca01 (Amir1)

24 May 2016

  • curprev 07:0307:03, 24 May 2016imported>Labslogbot 77,303 bytes +75 rebooting deployment-tin, can't log in (mobrovac)

23 May 2016

  • curprev 19:3519:35, 23 May 2016imported>Labslogbot 77,228 bytes +1,024 killed all mysqld process on Trusty CI slaves (hashar)

20 May 2016

  • curprev 20:3020:30, 20 May 2016imported>Labslogbot 76,204 bytes +151 Killing https://integration.wikimedia.org/ci/job/mediawiki-extensions-qunit/43608/ which has been running for 5 hours (bd808)

19 May 2016

  • curprev 16:4716:47, 19 May 2016imported>Labslogbot 76,053 bytes +778 deployment-tin jenkins worker seems to be back online after [https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update some prodding] (thcipriani)

18 May 2016

  • curprev 13:1613:16, 18 May 2016imported>Labslogbot 75,275 bytes +413 deploying a05e830 to ores nodes (sca01 and ores-web) (Amir1)

13 May 2016

  • curprev 14:3914:39, 13 May 2016imported>Labslogbot 74,862 bytes +601 remove shadow l10nupdate user from deployment-tin and mira in beta (thcipriani)

12 May 2016

  • curprev 22:5322:53, 12 May 2016imported>Labslogbot 74,261 bytes +86 Started dead mysql on integration-slave-precise-1011 (bd808)

11 May 2016

  • curprev 21:0521:05, 11 May 2016imported>Labslogbot 74,175 bytes +1,452 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/288128 #T134946 (hashar)

10 May 2016

  • curprev 19:1719:17, 10 May 2016imported>Labslogbot 72,723 bytes +1,609 beta / CI purging old Linux kernels: salt -v '*' cmd.run 'dpkg -l|grep ^rc|awk "{ print \$2 }"|grep linux-image|xargs dpkg --purge' (hashar)

9 May 2016

  • curprev 20:5820:58, 9 May 2016imported>Labslogbot 71,114 bytes +890 Unbroke puppet on integration-raita.integration.eqiad.wmflabs . Puppet was blocked because role::ci::raita was no more. Fixed by rebasing https://gerrit.wikimedia.org/r/#/c/208024 T115330 (hashar)

6 May 2016

  • curprev 19:4519:45, 6 May 2016imported>Labslogbot 70,224 bytes +172 Restart cassandra-metrics-collector on deployment-restbase0[1-2] (urandom)

5 May 2016

  • curprev 22:0922:09, 5 May 2016imported>Labslogbot 70,052 bytes +139 Promoted Yurik and Jgirault to sysops on beta enwiki. Through shell because logging in is broken for me. (MaxSem)

4 May 2016

  • curprev 21:2821:28, 4 May 2016imported>Labslogbot 69,913 bytes +1,227 deployed puppet FQDN domain patch for OCG: https://gerrit.wikimedia.org/r/286068 and restarted ocg on deployment-pdf0[12] (cscott)

3 May 2016

  • curprev 23:2123:21, 3 May 2016imported>Labslogbot 68,686 bytes +1,748 Changed "Maximum Number of Retries" for ssh agent launch in jenkins for deployment-tin from "0" to "10" (bd808)

2 May 2016

  • curprev 21:2721:27, 2 May 2016imported>Labslogbot 66,938 bytes +1,284 updated OCG to version b775e612520f9cd4acaea42226bcf34df07439f7 (cscott)

30 April 2016

  • curprev 18:3118:31, 30 April 2016imported>Labslogbot 65,654 bytes +136 deploying d4f63a3 from github.com/wiki-ai/ores-wikimedia-config into targets in beta cluster via scap3 (Amir1)

29 April 2016

28 April 2016

  • curprev 22:0322:03, 28 April 2016imported>Labslogbot 65,189 bytes +1,833 deployment-restbase01 upgrade to 2.2.6 complete : T126629 (urandom)

27 April 2016

  • curprev 23:5723:57, 27 April 2016imported>Labslogbot 63,356 bytes +1,474 nodepool instances running again after an openstack rabbitmq restart by andrewbogott (thcipriani)

26 April 2016

  • curprev 20:4520:45, 26 April 2016imported>Labslogbot 61,882 bytes +875 Regenerating Nodepool Jessie snapshot to include composer and HHVM | T128092 (hashar)
  • curprev 00:4600:46, 26 April 2016imported>Labslogbot 61,007 bytes +1,084 temporary keyholder fix in place in beta (thcipriani)

24 April 2016

22 April 2016

  • curprev 18:1318:13, 22 April 2016imported>Labslogbot 59,858 bytes +148 deploying https://gerrit.wikimedia.org/r/284841 (legoktm)

21 April 2016

  • curprev 19:0719:07, 21 April 2016imported>Labslogbot 59,710 bytes +234 scap version testing should be done, puppet should no longer be disabled on hosts (thcipriani)

20 April 2016

  • curprev 22:2822:28, 20 April 2016imported>Labslogbot 59,476 bytes +541 rolling back scap version in beta, legit failure :( (thcipriani)

17 April 2016

  • curprev 01:0101:01, 17 April 2016imported>Labslogbot 58,935 bytes +223 deploying https://gerrit.wikimedia.org/r/283837 (legoktm)

13 April 2016

  • curprev 01:4801:48, 13 April 2016imported>Labslogbot 58,712 bytes +83 deploying https://gerrit.wikimedia.org/r/282952 (legoktm)

12 April 2016

  • curprev 19:4719:47, 12 April 2016imported>Labslogbot 58,629 bytes +869 Cleaned up large hhbc cache file on deployment-medaiwiki03 via `sudo service hhvm stop; sudo rm /var/cache/hhvm/fcgi.hhbc.sq3; sudo service hhvm start` (bd808)

11 April 2016

8 April 2016

  • curprev 22:4622:46, 8 April 2016imported>Labslogbot 57,302 bytes +563 Created blobs1 table for all wiki DBs on Beta Cluster (matt_flaschen)

7 April 2016

  • curprev 20:1620:16, 7 April 2016imported>Labslogbot 56,739 bytes +217 deployment-mediawiki02.deployment-prep.eqiad.wmflabs , cleared up random left over stuff / big logs etc (hashar)

5 April 2016

  • curprev 23:5623:56, 5 April 2016imported>Labslogbot 56,522 bytes +561 Removed cherry-pick and rebased /var/lib/git/operations/puppet on integration-puppetmaster after merge of https://gerrit.wikimedia.org/r/#/c/281706/ (marxarelli)

4 April 2016

  • curprev 17:3017:30, 4 April 2016imported>Labslogbot 55,961 bytes +400 Phabricator going down in about 10 minutes to hopefully address the overheating issue: T131742 (greg-g)

3 April 2016

  • curprev 07:0207:02, 3 April 2016imported>Labslogbot 55,561 bytes +157 deploying https://gerrit.wikimedia.org/r/281079 (legoktm)

2 April 2016

  • curprev 22:5822:58, 2 April 2016imported>Labslogbot 55,404 bytes +425 added local hack to pupetmaster to make scap3 provider more verbose (Amir1)

1 April 2016

  • curprev 22:1622:16, 1 April 2016imported>Labslogbot 54,979 bytes +960 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/281046 (Krinkle)
  • curprev 00:1600:16, 1 April 2016imported>Labslogbot 54,019 bytes +413 created oathauth_users table on centralauth db in beta (csteipp)

30 March 2016

  • curprev 19:3219:32, 30 March 2016imported>Labslogbot 53,606 bytes +709 deleted some nutcracker and hhvm log files on deployment-mediawiki01 to free space (twentyafterfour)

29 March 2016

  • curprev 23:2223:22, 29 March 2016imported>Labslogbot 52,897 bytes +365 running jenkins-jobs update config/ 'mwext-donationinterfacecore125-testextension-zend53' to deploy https://gerrit.wikimedia.org/r/#/c/280261/ (marxarelli)

26 March 2016

25 March 2016

  • curprev 23:5523:55, 25 March 2016imported>Labslogbot 52,458 bytes +1,083 deleting instances integration-slave-trusty-1002 and integration-slave-trusty-1005 (marxarelli)

24 March 2016

  • curprev 16:3916:39, 24 March 2016imported>Labslogbot 51,375 bytes +350 restarted rsync service on deployment-tin (thcipriani)

23 March 2016

  • curprev 23:0023:00, 23 March 2016imported>Labslogbot 51,025 bytes +200 rm-rf integration-slave-trusty-1013:/mnt/home/jenkins-deploy/tmpfs/jenkins-2/karma-54925082/ (bad permissions, caused Karma issues) (Krinkle)

22 March 2016

  • curprev 17:4017:40, 22 March 2016imported>Labslogbot 50,825 bytes +83 deploying https://gerrit.wikimedia.org/r/278926 (legoktm)

21 March 2016

  • curprev 21:5521:55, 21 March 2016imported>Labslogbot 50,742 bytes +1,183 zuul: almost all MediaWiki extensions migrated to run the npm job on Nodepool (with Node.js 4.3) T119143 . All tested. Will monitor the build results that ran overnight tomorrow (hashar)

19 March 2016

  • curprev 13:0413:04, 19 March 2016imported>Labslogbot 49,559 bytes +114 Jenkins: added ldap-labs-codfw.wikimedia.org as a fallback LDAP server T130446 (hashar)

18 March 2016

17 March 2016

  • curprev 21:1021:10, 17 March 2016imported>Labslogbot 49,371 bytes +318 updating scap on deployment-tin to test D133 (thcipriani)

16 March 2016

  • curprev 14:2814:28, 16 March 2016imported>Labslogbot 49,053 bytes +143 Updated jobs having the package manager cache system (castor) via https://gerrit.wikimedia.org/r/#/c/277774/ (hashar)

15 March 2016

  • curprev 15:1715:17, 15 March 2016imported>Labslogbot 48,910 bytes +542 added wikidata.beta.wmflabs.org in https://wikitech.wikimedia.org/wiki/Special:NovaAddress to deployment-cache-text04.deployment-prep.eqiad.wmflabs (jzerebecki)

14 March 2016

  • curprev 22:1822:18, 14 March 2016imported>Labslogbot 48,368 bytes +398 new jobs weren't processing in Zuul, lego fixed it and blamed Reedy (greg-g)

11 March 2016

  • curprev 22:4222:42, 11 March 2016imported>Labslogbot 47,970 bytes +235 deploying https://gerrit.wikimedia.org/r/276901 (legoktm)

10 March 2016

  • curprev 20:2220:22, 10 March 2016imported>Labslogbot 47,735 bytes +503 Nodepool Image ci-jessie-wikimedia-1457641052 in wmflabs-eqiad is ready (hashar)

9 March 2016

  • curprev 17:5617:56, 9 March 2016imported>Labslogbot 47,232 bytes +2,402 Cleaned up git clone state in deployment-tin.deployment-prep:/srv/mediawiki-staging/php-master and queued beta-code-update-eqiad to try again (T129371) (bd808)

7 March 2016

  • curprev 21:0321:03, 7 March 2016imported>Labslogbot 44,830 bytes +226 Nodepool upgraded to 0.1.1-wmf.4 , it no more waits 1 minute before deleted a used node | T118573 (hashar)

6 March 2016

  • curprev 10:2010:20, 6 March 2016imported>Labslogbot 44,604 bytes +83 deploying https://gerrit.wikimedia.org/r/274911 (legoktm)

4 March 2016

  • curprev 19:3119:31, 4 March 2016imported>Labslogbot 44,521 bytes +435 Nodepool Image ci-jessie-wikimedia-1457119603 in wmflabs-eqiad is ready - T128846 (hashar)
  • curprev 00:1100:11, 4 March 2016imported>Labslogbot 44,086 bytes +947 salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" (Krinkle)

2 March 2016

  • curprev 16:2216:22, 2 March 2016imported>Labslogbot 43,139 bytes +297 reloading zuul for 9398fa1..943f17b (jzerebecki)
  • curprev 01:1901:19, 2 March 2016imported>Labslogbot 42,842 bytes +1,454 force restarting zuul because the queue is very stuck https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Restart (thcipriani)

29 February 2016

  • curprev 16:2316:23, 29 February 2016imported>Labslogbot 41,388 bytes +447 salt -v '*slave*' cmd.run 'rm -fR /mnt/jenkins-workspace/workspace/mwext*jslint' T127362 (hashar)

28 February 2016

  • curprev 01:3001:30, 28 February 2016imported>Labslogbot 40,941 bytes +119 Rebooting integration-slave-precise-1012 – Might help T109704 (MySQL not running) (Krinkle)

26 February 2016

  • curprev 15:1415:14, 26 February 2016imported>Labslogbot 40,822 bytes +1,342 salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" T128191 (jzerebecki)

25 February 2016

  • curprev 22:3822:38, 25 February 2016imported>Labslogbot 39,480 bytes +890 beta: maybe deployment-jobunner01 is processing jobs a bit faster now. Seems like hhvm went wild (hashar)
  • curprev 00:3100:31, 25 February 2016imported>Labslogbot 38,590 bytes +1,255 running puppet on beta to update scap to latest packaged version: sudo salt -b '10%' -G 'deployment_target:scap/scap' cmd.run 'puppet agent -t' (thcipriani)

23 February 2016

22 February 2016

  • curprev 22:0622:06, 22 February 2016imported>Labslogbot 37,039 bytes +498 Restarted puppetmaster service on deployment-puppetmaster to "fix" error "invalid byte sequence in US-ASCII" (bd808)

20 February 2016

  • curprev 20:1920:19, 20 February 2016imported>Labslogbot 36,541 bytes +415 beta-code-update-eqiad job repeatedly stuck at "IRC notifier plugin" (Krinkle)

19 February 2016

  • curprev 12:0912:09, 19 February 2016imported>Labslogbot 36,126 bytes +289 killed https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/ been running for 13 hours. Blocked because slave went offline due to labs reboots yesterday (hashar)

18 February 2016

  • curprev 19:2019:20, 18 February 2016imported>Labslogbot 35,837 bytes +492 deploying https://gerrit.wikimedia.org/r/271583 and https://gerrit.wikimedia.org/r/271581, both no-ops (legoktm)

17 February 2016

  • curprev 07:3607:36, 17 February 2016imported>Labslogbot 35,345 bytes +65 deploying https://gerrit.wikimedia.org/r/271201 (legoktm)
  • curprev 01:0101:01, 17 February 2016imported>Labslogbot 35,280 bytes +1,635 attempting to turn off NFS on 52 instances on deployment-prep project (yuvipanda)

15 February 2016

14 February 2016

13 February 2016

12 February 2016

  • curprev 23:5423:54, 12 February 2016imported>Labslogbot 31,361 bytes +908 beta cluster broken since 20:30 UTC https://logstash-beta.wmflabs.org/#/dashboard/elasticsearch/fatalmonitor havent looked (hashar)

11 February 2016

10 February 2016

  • curprev 23:5423:54, 10 February 2016imported>Labslogbot 27,938 bytes +3,058 depooling Trusty slaves that only have 2GB of ram that is not enough. https://phabricator.wikimedia.org/T126545 (hashar_)
  • curprev 01:0301:03, 10 February 2016imported>Labslogbot 24,880 bytes +3,895 deploying https://gerrit.wikimedia.org/r/269576 (legoktm)

9 February 2016

  • curprev 00:3900:39, 9 February 2016imported>Labslogbot 20,985 bytes +2,174 gallium edited /usr/share/python/zuul/local/lib/python2.7/site-packages/zuul/trigger/gerrit.py and modified: replication_timeout = 300 -> replication_timeout = 10 (hashar)

8 February 2016

  • curprev 00:5700:57, 8 February 2016imported>Labslogbot 18,811 bytes +87 deploying https://gerrit.wikimedia.org/r/268031 (legoktm[NE])

6 February 2016

5 February 2016

  • curprev 13:3013:30, 5 February 2016imported>Labslogbot 18,650 bytes +739 beta cleaning out /data/project/logs/archive was from pre logstash area. We no more log this way since May 2015 apparently (hashar)

4 February 2016

  • curprev 22:0822:08, 4 February 2016imported>Labslogbot 17,911 bytes +201 reloading zuul for bed7be1..f57b7e2 (jzerebecki)
  • curprev 00:5700:57, 4 February 2016imported>Labslogbot 17,710 bytes +2,091 Got deployment-bastion processing Jenkins jobs again via instructions left by my past self at https://phabricator.wikimedia.org/T72597#747925 (bd808)

2 February 2016

31 January 2016

  • curprev 13:3513:35, 31 January 2016imported>Labslogbot 13,530 bytes +453 Jenkins IRC bot started falling at Jan 30 01:04:00 2016 for whatever reason.... Should be fine now (hashar)

30 January 2016

  • curprev 12:4612:46, 30 January 2016imported>Labslogbot 13,077 bytes +110 integration-slave-jessie-1001 : fixed puppet.con server name and ran puppet (hashar)

29 January 2016

28 January 2016

26 January 2016

  • curprev 16:5116:51, 26 January 2016imported>Labslogbot 9,728 bytes +472 updated OCG to version 64050af0456a43344b32e3e93561a79207565eaf (cscott)

25 January 2016

  • curprev 18:5918:59, 25 January 2016imported>Labslogbot 9,256 bytes +525 started redis-server on deployment-redis01 by commenting out latency-monitor-threshold from the redis.conf (twentyafterfour)

24 January 2016

22 January 2016

  • curprev 23:5823:58, 22 January 2016imported>Labslogbot 8,583 bytes +1,274 removed skins from mwext-qunit workspace on trusty-1013 slave (legoktm)

21 January 2016

  • curprev 21:4121:41, 21 January 2016imported>Labslogbot 7,309 bytes +757 restored role::mail::mx on deployment-mx (hashar)
  • curprev 01:0001:00, 21 January 2016imported>Labslogbot 6,552 bytes +1,287 jenkins-deploy@deployment-bastion:/srv/mediawiki-staging/php-master/extensions$ git pull && git submodule update --init --recursive (jzerebecki)

19 January 2016

17 January 2016

16 January 2016

15 January 2016

14 January 2016

13 January 2016

  • curprev 21:0621:06, 13 January 2016imported>Labslogbot 214,628 bytes +353 beta cluster code is up to date again. Got delayed by roughly 4 hours. (hashar)

12 January 2016

  • curprev 23:3023:30, 12 January 2016imported>Labslogbot 214,275 bytes +277 deploying https://gerrit.wikimedia.org/r/263757 https://gerrit.wikimedia.org/r/263756 (legoktm)

11 January 2016

  • curprev 22:2522:25, 11 January 2016imported>Labslogbot 213,998 bytes +501 Deleting old references on Zuul-merger for mediawiki/core : <tt>/usr/share/python/zuul/bin/python /home/hashar/zuul-clear-refs.py --until 15 /srv/ssd/zuul/git/mediawiki/core </tt> (hashar)

7 January 2016

  • curprev 23:1623:16, 7 January 2016imported>Labslogbot 213,497 bytes +290 deleted /mnt/jenkins-workspace/workspace/mediawiki-extensions-qunit/src/extensions/PdfHandler/.git/refs/heads/wmf/1.26wmf16.lock on slave 1013 (legoktm)
  • curprev 01:2501:25, 7 January 2016imported>Labslogbot 213,207 bytes +835 reloading zuul for b0a5335..c16368a (jzerebecki)

5 January 2016

  • curprev 16:5416:54, 5 January 2016imported>Labslogbot 212,372 bytes +322 Removed elastic search from CI slaves https://phabricator.wikimedia.org/T89083 https://gerrit.wikimedia.org/r/#/c/259301/ (hashar_)

4 January 2016

2 January 2016

1 January 2016

  • curprev 22:1722:17, 1 January 2016imported>Labslogbot 211,782 bytes +121 No nodepool ci-jessie-* hosts seen in Jenkins interface and rake-jessie jobs backing up (bd808)

30 December 2015

  • curprev 00:1300:13, 30 December 2015imported>Labslogbot 211,661 bytes +779 rake-jessie jobs running again which will hopefully clear the large zuull backlog (bd808)

26 December 2015

  • curprev 21:4121:41, 26 December 2015imported>Labslogbot 210,882 bytes +618 integration: getting rid of $wgHTTPProxy https://gerrit.wikimedia.org/r/261096 (no more needed) (hashar)

23 December 2015

  • curprev 23:3723:37, 23 December 2015imported>Labslogbot 210,264 bytes +192 Reloading Zuul to deploy I39b9f292e95363addf8983eec5d08a0af527a163 (marxarelli)

22 December 2015

  • curprev 23:3123:31, 22 December 2015imported>Labslogbot 210,072 bytes +496 (when npm jobs run) - sudo rm -rf /mnt/home/jenkins-deploy/.npm at integration-slave-trusty-1015 (due to cache corruption) (Krinkle)

21 December 2015

19 December 2015

  • curprev 22:1822:18, 19 December 2015imported>Labslogbot 209,035 bytes +412 sudo rm -rf /mnt/home/jenkins-deploy/tmpfs/jenkins* on integration-slave-precise-1014 (Krinkle)
  • curprev 01:2301:23, 19 December 2015imported>Labslogbot 208,623 bytes +1,774 Added Reedy as a projectadmin in the integration project (bd808)

17 December 2015

  • curprev 23:0723:07, 17 December 2015imported>Labslogbot 206,849 bytes +1,799 sudo salt --show-timeout '*slave*' cmd.run 'rm -fR /mnt/home/jenkins-deploy/tmpfs/jenkins-?/*' (legoktm)

16 December 2015

  • curprev 23:2323:23, 16 December 2015imported>Labslogbot 205,050 bytes +877 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/259616 (Krinkle)

15 December 2015

  • curprev 22:5722:57, 15 December 2015imported>Labslogbot 204,173 bytes +881 On scandium created /srv/ssd/zuul/git/wikimedia/fundraising/crm repo manually. Namespace conflict with wikimedia/fundraising/crm/civicrm.git which prevented zuul-merger to clone the arm repo (hashar)
  • curprev 00:5700:57, 15 December 2015imported>Labslogbot 203,292 bytes +613 marking integration-slave-trusty-1012 offline, strange zuul.cloner behavior. (thcipriani)

13 December 2015

  • curprev 23:3023:30, 13 December 2015imported>Labslogbot 202,679 bytes +171 Ran deployment-bastion:~bd808/cleanup-var-crap.sh and freed 846M on /var (bd808)

11 December 2015

  • curprev 22:1822:18, 11 December 2015imported>Labslogbot 202,508 bytes +909 Stopped zuul merger on gallium to have phabricator/extensions populated on scandium (namespacing issue). Restarted zuul-merger on gallium once done. (hashar)
  • curprev 01:1201:12, 11 December 2015imported>Labslogbot 201,599 bytes +945 deploying https://gerrit.wikimedia.org/r/258395 (legoktm)

10 December 2015

8 December 2015

  • curprev 20:3120:31, 8 December 2015imported>Labslogbot 199,584 bytes +1,662 beta cluster instances switching to new ldap configuration (hashar)

7 December 2015

  • curprev 16:1616:16, 7 December 2015imported>Labslogbot 197,922 bytes +790 Nodepool no more listens for Jenkins events over ZeroMQ. No TCP connection established on port 8888 (hashar)

4 December 2015

3 December 2015

  • curprev 23:5323:53, 3 December 2015imported>Labslogbot 196,922 bytes +509 Reloading Zuul to deploy If60f720995dfc7859e53cf33043b5a21b1a4b085 (marxarelli)

2 December 2015

  • curprev 11:1611:16, 2 December 2015imported>Labslogbot 196,413 bytes +556 configure wmf-insecte to join #wikimedia-android-ci ( https://gerrit.wikimedia.org/r/#/c/254905/3/jjb/mobile.yaml,unified ) (hashar)

1 December 2015

  • curprev 16:0616:06, 1 December 2015imported>Labslogbot 195,857 bytes +584 split mediawiki core parser tests under Zend to their own job https://gerrit.wikimedia.org/r/#/c/256006/ (hashar)

29 November 2015

  • curprev 23:2323:23, 29 November 2015imported>Labslogbot 195,273 bytes +436 updated cherry-pick of https://gerrit.wikimedia.org/r/#/c/255916/ to PS2 (bd808)

27 November 2015

  • curprev 19:2519:25, 27 November 2015imported>Labslogbot 194,837 bytes +235 Trebuchet returners not reporting to redis from sca0[12] (bd808)

26 November 2015

  • curprev 22:3522:35, 26 November 2015imported>Labslogbot 194,602 bytes +553 Upgraded Zuul on labs Precise hosts to zuul_2.1.0-60-g1cc37f7-wmf4precise1 ( https://phabricator.wikimedia.org/T119741 ) (hashar)

25 November 2015

  • curprev 19:5319:53, 25 November 2015imported>Labslogbot 194,049 bytes +173 ran mwscript sql.php --wiki=enwiki --wikidb=wikishared /srv/mediawiki-staging/php-master/extensions/Echo/db_patches/echo_unread_wikis.sql (legoktm)

24 November 2015

  • curprev 21:1621:16, 24 November 2015imported>Labslogbot 193,876 bytes +378 bumping our JJB fork to 4eb33b2. Pinning jenkins==0.4.8 with https://gerrit.wikimedia.org/r/255182 (hashar)

23 November 2015

  • curprev 22:3222:32, 23 November 2015imported>Labslogbot 193,498 bytes +1,069 Updating parsoidsvc-source-npm-* jobs https://gerrit.wikimedia.org/r/254946 (hashar)

22 November 2015

  • curprev 21:1521:15, 22 November 2015imported>Labslogbot 192,429 bytes +110 Zuul queue stuck as of 3 hours ago. Jenkins unresponsive or 503 over HTTP. (Krinkle)

20 November 2015

  • curprev 21:5921:59, 20 November 2015imported>Labslogbot 192,319 bytes +266 Removed shadow l10nupdate user from /etc/passwd & /etc/shadow on mira.deployment-prep.eqiad.wmflabs (bd808)

13 November 2015

  • curprev 21:1121:11, 13 November 2015imported>Labslogbot 192,053 bytes +992 zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/tools/releng --change 200240,1 (hashar)

12 November 2015

  • curprev 22:0322:03, 12 November 2015imported>Labslogbot 191,061 bytes +94 rm -rf mediawiki-extensions-qunit workspace on trusty-1012 (legoktm)

11 November 2015

  • curprev 17:5817:58, 11 November 2015imported>Labslogbot 190,967 bytes +56 reloading zuul for e81ee95..acbd851 (jzerebecki)
  • curprev 00:0200:02, 11 November 2015imported>Labslogbot 190,911 bytes +500 deployment-puppetmaster:/var/lib/git/operations/puppet was left mid-rebase conflicts with portals apache config. Fixed conflicts added the file and continued the rebase. (thcipriani)

9 November 2015

  • curprev 23:1923:19, 9 November 2015imported>Labslogbot 190,411 bytes +630 put https://integration.wikimedia.org/ci/computer/integration-slave-trusty-1015/ back on (hashar)

7 November 2015

6 November 2015

5 November 2015

4 November 2015

  • curprev 23:1823:18, 4 November 2015imported>Labslogbot 188,479 bytes +820 Running `jenkins-jobs update config/ 'mw-tools-scap-tox-doc-publish'` to test Ib4753ad493115682d902cf15136199fd2083b8e5 (marxarelli)

3 November 2015

  • curprev 22:4822:48, 3 November 2015imported>Labslogbot 187,659 bytes +690 Reloading Zuul to deploy https://gerrit.wikimedia.org/r/250847 (thcipriani)

2 November 2015

  • curprev 20:4820:48, 2 November 2015imported>Labslogbot 186,969 bytes +1,694 Upgraded Zuul on Trusty slaves to /root/zuul_2.1.0-60-g1cc37f7-wmf1trusty1_amd64.deb (hashar)

1 November 2015

31 October 2015

  • curprev 21:3221:32, 31 October 2015imported>Labslogbot 185,192 bytes +107 Ran `sudo rm /var/log/account/pacct.?` on deployment-bastion; freed ~425M (bd808)

30 October 2015

  • curprev 18:2018:20, 30 October 2015imported>Labslogbot 185,085 bytes +401 deployment-db2 replication recovered after slave stop/reset/set master position/start (marxarelli)

29 October 2015

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)