Nova Resource:Library-upgrader/SAL

2021-11-08

• 05:21 legoktm: deleting libup-db01, replaced by trove

2021-11-06

• 22:03 legoktm: deleting libup-web01 instance
• 22:03 legoktm: disabled libup-web unit on `upgrader-06` instance
• 22:00 legoktm: https://libraryupgrader2.wmcloud.org/ now points to libup-web02, which is fully containerized and auto-deployed via the deployment pipeline
• 21:35 legoktm: launching libup-web02 (bullseye) to replace libup-web01
• 21:26 legoktm: shutting down libup-db01, will delete in a few days
• 20:58 legoktm: launching libup-db02 via trove to replace libup-db01

2021-10-25

• 17:00 legoktm: temporarily stopped libup-celery service to test new monitoring/alerting

2021-08-25

• 08:26 legoktm: restarted failed libup-push systemd unit and cleared queue to avoid pushing really old patches

2021-04-15

• 09:44 arturo: bumping quota: 16->20GB RAM, 8->10 CPUs (T280103)

2021-04-09

• 08:12 legoktm: importing mysql database on libup-db01
• 07:34 legoktm: creating libup-db01 instance
• 07:32 legoktm: deleting libup-diff01 experiment and upgrader-07 (puppet tests) instances

2021-03-05

• 05:01 legoktm: sudo systemctl enable libup-push
• 04:11 legoktm: added SSH key to ssh-agent

2020-10-26

• 23:51 legoktm: added key to ssh-agent and started libup-celery

2020-09-03

• 21:50 legoktm: added libraryupgrader2.wmcloud.org DNS proxy and remove wmflabs.org one for automatic redirect (T261995)

2020-06-23

• 02:52 legoktm: created libup-diff01 instance
• 02:41 legoktm: deleted upgrader-05 instace, which has been shutoff for a while now

2020-03-24

• 06:33 legoktm: building fresh docker image

2020-01-10

• 09:34 legoktm: shutdown upgrader-05 instance
• 07:52 legoktm: switched over libraryupgrader2.wmflabs.org to point to upgrader06
• 06:59 legoktm: upgrader06: adduser libup && adduser libup docker
• 06:59 legoktm: upgrader06: enabling buster-wikimedia/thirdparty/kubeadm-k8s apt repo and installing docker-ce
• 06:53 legoktm: disabling all services on upgrader-05
• 06:33 legoktm: creating upgrader-06 instance, large flavor this time

2019-12-28

• 23:49 legoktm: deleting upgrader-04 instance (T236559)

2019-12-20

• 01:14 legoktm: moved a lot of old logs into /srv/data/archive/

2019-12-14

• 22:56 legoktm: dropped concurrency down to 1
• 22:52 legoktm: restarted libup-celery now that gerrit-replica has been restarted
• 20:06 legoktm: systemctl stop libup-celery # because of T240763
• 11:47 legoktm: temporarily bump concurrency to 2 to clear backlog

2019-10-13

• 22:45 legoktm: rebooting upgrader-05 because of DNS failures when run in docker

2019-07-31

• 04:16 legoktm: added ssh key to libraryupgrader Gerrit account that lives on upgrader-05

2019-06-21

• 04:49 legoktm: set up new upgrader-05 stretch instance (using docker from upstream docker's repo :((), using run.py as a stress test

2019-06-11

• 15:03 legoktm: -04 has a full disk, and is broked. oops.

2018-11-14

• 15:09 andrewbogott: migrating project to eqiad1-r