Release Engineering/SAL

2020-02-28

• 20:00 brennen: finished deleting wmf/1.34.0-wmf.* branches for T244368
• 19:37 brennen: in process of deleting many old branches for T244368
• 12:40 hauskatze: Migrated https://github.com/wikimedia/mediawiki-extensions-NearbyPages to Gerrit; GitHub repo now watches the Gerrit one
• 00:18 James_F: jjb: Deleting wikidata-query-gui-build

2020-02-27

• 18:11 James_F: Reset mediawiki-quibble-composer-mysql-php72-docker to nominal T246358
• 17:59 James_F: Updating mediawiki-quibble-composer-mysql-php72-docker to test probably-breaking requirement that error logs are empty T246358
• 15:53 James_F: Docker: Publish mediawiki-phan-testrun:0.1.3 for T226117
• 00:56 Krenair: Added edtadros to deployment-prep per request on IRC and https://en.wikipedia.org/w/index.php?title=User:ETadros_(WMF)/sandbox&diff=942815687&oldid=901073254 - wants access to eventlog05 in particular

2020-02-26

• 21:45 James_F: Docker: Publishing mediawiki-phan-testrun:0.1.0 for T226117
• 17:12 James_F: Zuul: Set up CI for mediawiki/extensions/MediaModeration T246243
• 00:17 James_F: Zuul: Add a job to test new phan changes on MW core T226117

2020-02-25

• 23:18 Krenair: Added mayakpwiki to deployment-prep, user is already in the nda group and is working for WMF Product Analytics. Wants access to eventlog05
• 23:04 Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/574890
• 21:00 hauskatze: gerrit: replication start mediawiki/extensions/MediaModeration --wait # New repo
• 20:43 James_F: Running a manual sync for Beta Cluster.
• 20:17 James_F: Beta jobs on Jenkins backed up; doing the restart dance.
• 19:07 James_F: Ran `sudo -u doc-uploader git -C /srv/docroot pull` on doc1001.
• 16:52 James_F: Zuul: [mediawiki/extensions/Popups] Publish PHP documentation T242779
• 13:34 godog: set number of replicas for es indices to 0 on deployment-logstash03

2020-02-24

• 20:59 James_F: doc1001: `sudo -u doc-uploader rm /srv/docroot/org/wikimedia/doc/MinervaNeue/master` except PHP bits from Doxygen; everything else is at mediawiki-skins-MinervaNeue/ now
• 20:45 James_F: Zuul: [mediawiki/skins/Vector] Publish PHP documentation T242779
• 18:12 brennen: restarted jenkins on releases1001 and releases2001
• 15:23 godog: cherry-pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/574485
• 13:50 Lucas_WMDE: canceled one beta-scap-eqiad job per https://w.wiki/J5$
• 13:44 Urbanecm: deployment-prep: Run scap sync manually, it doesn't seem it was invoked by jenkins for some reason

2020-02-21

• 23:27 James_F: Zuul: Allow l10n on analytics/wikistats2 for T245805
• 19:38 thcipriani: remove a bunch of extensions from beta that were on disk but not in https://gerrit.wikimedia.org/rg/mediawiki/extensions/
• 01:02 James_F: Zuul: [operations/mediawiki-config] Run tox-docker always
• 00:59 James_F: Zuul: [operations/mediawiki-config] Run tox-docker as experimental

2020-02-20

• 22:08 thcipriani: killing a whole bunch of backed-up beta-scap-eqiad jobs
• 21:50 James_F: deployment-deploy01 back online; Jenkins backlog clearing for beta cluster jobs.
• 21:47 James_F: Taking deployment-deploy01 temporarily offline
• 19:44 hauskatze: GitHub: RENAMED https://github.com/wikimedia/operations-software-atskafka to https://github.com/wikimedia/atskafka # T245731
• 19:43 hauskatze: GitHub: Created https://github.com/wikimedia/operations-software-atskafka # T245731
• 19:42 hauskatze: replication start operations/software/atskafka --wait
• 10:04 Reedy: postmerge jobs for mediawiki-config look stuck
• 10:03 Reedy: postmerge jobs look stuck

2020-02-19

• 18:58 thcipriani: unblocking beta-scap-eqiad job (stuck for 18 hrs)
• 16:45 brennen: Updating dev-images docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/572488

2020-02-18

• 22:01 brennen: adding dpifke to Gerrit Managers
• 21:23 James_F: Zuul: [mediawiki/extensions/Teahouse] Archive repo T176349

2020-02-17

• 12:34 hauskatze: extension-Teahouse@phab: Mirror deactivated # T176349
• 12:34 hauskatze: extension-Teahouse@github: Mirror deleted # T176349

2020-02-16

• 21:24 hauskatze: AbuseFilterBypass@github: Deleted 'dev' branch | T245363
• 21:24 hauskatze: AbuseFilterBypass@github: Switch default branch of mirror to 'master' | T245363

2020-02-15

• 23:52 legoktm: rebuilding tox docker image for https://gerrit.wikimedia.org/r/572453
• 22:21 legoktm: reloading zuul for https://gerrit.wikimedia.org/r/572423
• 21:33 hauskatze: GitHub: Activated Travis CI at https://travis-ci.org/wikimedia/texvcjs for https://github.com/wikimedia/texvcjs per https://github.com/wikimedia/texvcjs/pull/34 and T245344
• 20:25 Krenair: T243226 Shut off deployment-puppetdb02 and deployment-puppetmaster03 - will leave for a week before deletion

2020-02-14

• 21:53 andrewbogott: moving deployment-puppetdb02 and deployment-puppetmaster03 off of cloudvirt1014 (which will be drained next week anyway)
• 20:33 brennen: Updating dev-images docker-pkg files on contint1001 to attempt a rebuild for https://gerrit.wikimedia.org/r/c/releng/dev-images/+/571828
• 20:05 thcipriani: deployment-mwmaint01:rm lingering .~tmp~ directories from failed rsync's taking up space
• 19:31 thcipriani: deployment-snapshot01:sudo apt-get clean cleared 1.2G of space
• 17:34 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/571849
• 17:23 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/571849
• 17:20 thcipriani: updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/571849

2020-02-13

• 22:19 James_F: Zuul: Add *.microsoft.com in email_whitelist
• 17:10 thcipriani: truncate deployment-puppetmaster04:/var/log/daemon.log -> 5G, 100% disk usage
• 15:58 brennen: Updating dev-images docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/releng/dev-images/+/571828

2020-02-12

• 21:36 thcipriani: reloading zuul in an attempt to troubleshoot T244690
• 20:22 James_F: Depooling integration-agent-docker-1001; it's being odd T245054
• 17:11 brennen: restarting jenkins for updates
• 01:24 brennen: Updating dev-images docker-pkg files on contint1001 for T228910

2020-02-11

• UKInEU: ssh -p 29418 paladox@gerrit.wikimedia.org replication start wikibase/vuejs-components --wait
• 21:27 hauskatze: github: Created https://github.com/wikimedia/wikibase-vuejs-components | T240224
• 18:24 marxarelli: 1.35.0-wmf.19 was branched at 3ac2e9f for T233867
• 16:44 James_F: Zuul: [mediawiki/extensions/Echo] Add experimental Postgres job
• 16:17 James_F: Terminated two multi-day running "parsoid-pipeline-publish" jobs
• 09:46 Daimona: foreachwiki extensions/AbuseFilter/maintenance/fixOldLogEntries.php | T228655
• 09:43 Daimona: mwscript extensions/AbuseFilter/maintenance/fixOldLogEntries.php --wiki=enwiki | T228655

2020-02-10

• 23:42 James_F: Zuul: [parsoid/deploy] Make PHP tests voting T239642
• 23:28 legoktm: restarting zuul
• 22:24 James_F: Zuul: Add tox for operations/homer/mock-private T244690
• 19:48 James_F: Add mahveotm to CI whitelist
• 16:20 Daimona: mwscript extensions/AbuseFilter/maintenance/fixOldLogEntries.php --wiki=dewiki | T228655
• 09:42 legoktm: reloading zuul for https://gerrit.wikimedia.org/r/571254
• 09:39 legoktm: building php74-compile image https://gerrit.wikimedia.org/r/542071
• 09:25 legoktm: building php74-compile image https://gerrit.wikimedia.org/r/571251
• 09:11 legoktm: building php74-compile image https://gerrit.wikimedia.org/r/571247
• 09:02 legoktm: reloading zuul for https://gerrit.wikimedia.org/r/571242
• 08:44 legoktm: reloading zuul for https://gerrit.wikimedia.org/r/539724
• 08:30 legoktm: reloading zuul for https://gerrit.wikimedia.org/r/571236
• 08:17 legoktm: updating tox-mysqld docker-pkg image https://gerrit.wikimedia.org/r/571234
• 07:43 legoktm: updating tox docker-pkg image for gettext https://gerrit.wikimedia.org/r/571019
• 07:20 legoktm: updating tox docker-pkg image (try #2) https://gerrit.wikimedia.org/r/571127
• 07:00 legoktm: updating docker-pkg for https://gerrit.wikimedia.org/r/571017

2020-02-07

• 23:20 brennen: Updating dev-images docker-pkg files on contint1001 for T244382
• 23:20 dpifke: puppetdb on deployment-puppetdb03 was killed by kernel OOM at Feb 7 09:50:29, per syslog. I just ran `systemctl start puppetdb` on that host, to fix puppet issues in beta.
• 19:10 James_F: Zuul: [labs/tools/video-cut-tool-back-end] Add node10 CI T244079
• 19:01 James_F: Zuul: [labs/tools/VideoCutTool] Switch from tox to node T244079
• 16:57 halfak: deploying ores a6f4f14

2020-02-06

• 20:27 James_F: Zuul: Run WikibaseCirrusSearch tests for every change in CirrusSearch T221135

2020-02-05

• 23:39 brennen: Updating dev-images docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/c/releng/dev-images/+/568140
• 16:56 thcipriani: Updating docker-pkg files on contint100 to deploy https://gerrit.wikimedia.org/r/566915
• 16:40 thcipriani: restarting zuul

2020-02-04

• 23:28 Krinkle: Puppet agent has not been able to run on deployment-mediawiki-07 for 2.5 days since 2 Feb 18:27
• 01:56 James_F: Zuul: [wikimedia/portals] Add service-pipeline configuration T238747
• 01:42 James_F: Zuul: [mediawiki] Run mediawiki-quibble-apitests-vendor-docker always T243975

2020-02-03

• 22:53 James_F: Zuul: Add CI for labs/tools/VideoCutTool T244079
• 22:36 James_F: Zuul: New email address for Ricordisamoa
• 21:44 James_F: Zuul: [mediawiki] Run mediawiki-quibble-apitests-vendor-docker experimentally T243975
• 16:43 hauskatze: Gerrit: replication start mediawiki/extensions/AControlImageLink --wait # new repo
• 16:28 James_F: Zuul: Publish `npm run doc` for Vector T242674

2020-02-02

• afk: deployment-deploy01$ while read host; do ssh-keyscan -H "$host"; done < <(grep -vP '^#' /etc/dsh/group/mediawiki-installation | grep -vP '^$' ) | sudo -u jenkins-deploy tee /mnt/home/jenkins-deploy/.ssh/known_hosts

2020-01-31

• 11:47 Lucas_WMDE: lucaswerkmeister-wmde@deployment-deploy01:~$ mwscript createAndPromote.php enwiki 'Lucas Werkmeister (WMDE)' --force --custom-groups=bureaucrat # make myself bureaucrat to make colleagues sysops
• 00:48 James_F: Restarting Jenkins.

2020-01-29

• 23:55 James_F: zuul: [data-values/value-view] Migrate to node10, now that that passes T228453
• 20:56 Krinkle: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/568600
• 00:05 James_F: layout: [mediawiki/extension/NSFileRepo] Restore tests now they're fixed T196480

2020-01-26

• 23:23 James_F: Zuul: Mark mediawiki/extensions/BlueSpiceMenues as archived T240788
• 22:20 James_F: Zuul: Mark mediawiki/extensions/SemanticTasks as archived T221549
• 18:51 hauskatze: maurelio@deployment-deploy01:~$ foreachwiki maintenance/emptyUserGroup.php ep[campus,online,coordinator,instructor] | T241504
• 11:02 hauskatze: github - deleted mirror for extension-BlueSpiceMenues; refs. T240788

2020-01-24

• 21:15 halfak: deploying ores 283f627
• 01:52 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/566911
• 01:45 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/566910
• 00:08 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/566895
• 00:02 James_F: Taking deployment-deploy01 offline to fix beta config update deadlock.

2020-01-23

• 23:43 thcipriani: Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/562365 and ttps://gerrit.wikimedia.org/r/562364
• 21:59 James_F: Drop REL1_32 testing, jobs, and pipelines, EOLed T242981
• 21:50 halfak: deploying ores 039251f (reverting to last good state)
• 17:17 James_F: layout: [mediawiki/extensions/TagContent] Archive repo T243093
• 15:31 halfak: deploying ores 283f627a

2020-01-22

• 20:19 James_F: Zuul: Adding TextExtracts as a dependency for TwitterCards T242171
• 19:37 Krinkle: Restore php-apcu version on deployment-mediawiki-07 to …wmf1, ref T236800
• 19:09 Krinkle: Installing newer version of php-apcu on deployment-mediawiki-07, ref T236800

2020-01-21

• 22:10 Krenair: deployment-prep added jbond as admin, T243226
• 22:09 mutante: gerrit - rsyncing 'git' and 'plugin' data dirs and /var/lib/gerrit2/review_site/ from gerrit1001 to gerrit1002 WITH --delete T239151
• 21:48 mutante: gerrit - rsyncing git data from gerrit1001 to gerrit1002 (T239151)
• 19:16 marxarelli: updating 105 jobs affected by https://gerrit.wikimedia.org/r/c/integration/config/+/566332

2020-01-20

• 21:24 Krinkle: fix Jenkins Console Section patterns to account for change in Quibble output (no more colors and now "cmd: << Start/Finish x" instead of "cmd:x finished"
• 10:48 addshore: reload zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/566001/ (node10 wikidata query gui)
• 10:10 addshore: the last zuul reload also pulled in "jjb: [java8-sonar-scanner] version bump to 0.5.4" .. T238004
• 10:10 addshore: reload zuul for https://gerrit.wikimedia.org/r/#/c/integration/config/+/565985/

2020-01-17

• 19:52 James_F: Docker: Publishing java8-sonar-scanner:0.5.4 on contint1001 T238004
• 19:25 James_F: Zuul: [operations/debs/hhvm] Archive T237038
• 19:01 James_F: Zuul: [AbuseFilterBypass] Install quibble for initial CI
• 18:53 James_F: Zuul: [skin-quibble] Disable PHP74 in gate for now T242980
• 18:30 James_F: Zuul: [WebAuthn] Add phan dependency on OATHAuth
• 18:27 James_F: Zuul: [WikiSEO] Add phan and seccheck jobs
• 18:24 James_F: Zuul: [ShortDescription] Add phan and seccheck jobs
• 18:21 James_F: Zuul: [NativeSvgHandler] Add phan and seccheck jobs
• 18:19 James_F: Zuul: [Sofa] Add phan and seccheck jobs
• 01:31 Reedy: apt-get clean and purged some old kernels from deployment-logstash2 to free up ~1.7G space

2020-01-16

• 20:09 James_F: Zuul: Add Aron Manning to CI whitelist
• 16:37 longma: bringing integration-slave-docker-1003 back online
• 16:30 longma: restarting integration-agent-docker-1003
• 16:24 longma: s/slave/agent
• 16:22 longma: bringing integration-slave-docker-1006 back online
• 16:15 longma: integration-slave-docker-1006:/# docker rmi $(docker images | grep " months " |grep -v " [1-2] months " | awk '{print $3}')
• 14:28 Reedy: killed beta job to unblock beta-scap et al
• 09:31 Amir1: "delete from wikidatawiki.wb_terms where term_full_entity_id like 'P%'" (T219301 T225054)

2020-01-15

• 23:00 Reedy: deleted `contint2001:/srv/zuul/git/mediawiki/services/change-propagation` as the repo seems to be corrupted (git remote was integration/docroot) like T134062
• 22:43 Reedy: deleted `contint1001:/srv/zuul/git/mediawiki/services/change-propagation` as the repo seems to be corrupted (git remote was integration/docroot) like T134062
• 15:50 thcipriani: reloading zuul to deploy https://gerrit.wikimedia.org/r/565042
• 01:15 marxarelli: deploy zuul layout change https://gerrit.wikimedia.org/r/c/integration/config/+/564813

2020-01-14

• 21:44 James_F: Zuul: Stop running JS tests for Parsoid and Parsoid-deploy T242782
• 19:38 Krinkle: No syslog entries from php-fpm or puppet-agent on deployment-mediawiki-07, T242659, rebooting server
• 18:02 James_F: Zuul: Add JeremyNguyen to Jenkins whitelist T235286
• 17:54 James_F: Zuul: [mediawiki/core] Stop publishing docker images for now T242775
• 15:33 James_F: Depooling integration-agent-docker-1003 due to issues.

2020-01-13

• 19:34 James_F: Zuul: [mediawiki/extensions/NativeSvgHandler] Add basic CI via quibble
• 16:50 James_F: Docker: Publishing java7-sonar-scanner:0.5.3 on contint1001

2020-01-10

• 23:22 James_F: Zuul: Restore extension-coverage to projects with …-codehealth T242444
• 18:35 bd808: Restarted zuul on contint1001 at 18:29Z; no logs since 2020-01-10 17:55:28,452

2020-01-09

• 17:26 James_F: Zuul: Run sonar scanner analysis in codehealth pipeline for select Java projects T238004
• 17:04 James_F: Docker: Publishing java7-sonar-scanner:0.5.2
• 16:29 James_F: Docker: Publish PHP 7.2.26-based images for T241223

2020-01-08

• 20:56 James_F: Docker: Publishing Quibble 0.0.40 images on contint1001 T192167 T220586 T236222 T236680
• 20:42 James_F: Terminating some jobs Zuul doesn't know about.
• 20:41 mutante: contint1001 - restartted zuul service
• 20:39 James_F: Safe-restarting Jenkins to see if it fixes the lack of progress
• 19:57 thcipriani: root@integration-cumin:~# cumin -b1 'name:integration-agent-docker' 'sleep 1; git -C /srv/git/mediawiki/core.git fetch'
• 19:25 James_F: Tagged Quibble 0.0.40 @ ba365ab T192167 T220586 T236222 T236680
• 16:26 halfak: deploiying ores 039251f

2020-01-07

• 18:18 James_F: Whitelist: Add RetroCraft and Ostrzyciel
• 18:16 James_F: Zuul: [mediawiki/extensions/SparkPost] Run phan
• 09:02 twentyafterfour: deploying https://gerrit.wikimedia.org/r/c/integration/config/+/562417

2020-01-06

• 20:43 James_F: Deleting https://doc.wikimedia.org/Wikibase/wmf/1.34.0-wmf.4 for T240206
• 20:43 James_F: Deleting https://doc.wikimedia.org/Wikibase/wmf/1.33.0-wmf.23 for T240206
• 20:11 James_F: Zuul: [SoftRedirector] Add phan and seccheck
• 19:45 James_F: Zuul: Add CI for new event schema repos T240985
• 17:24 James_F: Zuul: Added ptrcnull to the CI whitelist
• 17:05 James_F: Zuul: [labs/tools/indic-wscontest] Provide CI with tox-docker T241537

2020-01-02

• 18:19 hauskatze: maurelio@deployment-deploy01:~$ foreachwiki maintenance/emptyUserGroup.php translate-proofr | T241504
• 18:16 hauskatze: maurelio@deployment-deploy01:~$ foreachwiki maintenance/emptyUserGroup.php afttest-hide | T241504
• 18:15 hauskatze: maurelio@deployment-deploy01:~$ foreachwiki maintenance/emptyUserGroup.php afttest | T241504
• Away: Archive RecentActivityFeed extension T241032
• Away: Update Revi's e-mail on the whitelist
• Away: Add Safan41 to the whitelist T235286
• Away: Add BrandonXLF to the whitelist T235286
• 10:40 dcausse: created missing elastic indices: T241487

Archives

• Archive 1 (September 2014 - December 2015)
• Archive 2 (2016)
• Archive 3 (2017)
• Archive 4 (2018)
• Archive 5 (2019)
• Archive 6 (2020)
• Archive 7 (2021)
• Archive 8 (2022)