Nova Resource:Video/SAL

2017-06-26

• 23:56 zhuyifei1999_: oops, depooled encoding02 instead
• 23:54 zhuyifei1999_: depool encoding03
• 23:50 zhuyifei1999_: Doing that again, should be finally working
• 23:34 zhuyifei1999_: Doing that again :(
• 23:24 zhuyifei1999_: Fixing tmpfiles configuration on encoding0{1..3}, and rebooting; expecing to see workers back alive after reboot
• 23:16 zhuyifei1999_: failed to start because systemd-tmpfiles-setup failed. investigating
• 23:00 zhuyifei1999_: starting v2c workers at encoding02 & 03, somehow they didn't start automatically after reboot

2017-06-21

• 00:04 zhuyifei1999_: depool v2c worker on encoding01,02; repool encoding01,03
• 00:03 zhuyifei1999_: `sudo -H pip install -U youtube-dl` on all encoding hosts

2017-06-11

• 14:25 zhuyifei1999_: Restart v2c frontend
• 14:24 zhuyifei1999_: Repool v2c on encoding0{1,2}
• 14:23 zhuyifei1999_: Depool v2c on encoding0{2,3}
• 14:15 zhuyifei1999_: Updating code on encoding0{1..3} and frontend to 1e60ecc

2017-03-27

• 13:49 zhuyifei1999_: Disable autoconfirmed + 50 edits requirement in v2c for GSoC demonstration

2017-02-17

• 11:03 zhuyifei1999_: git pulling v2c frontend to 6f329f3 and restarting webservice

2017-02-16

• 11:59 zhuyifei1999_: depooled v2c worker at encoding01, depooled & repooled encoding02, repooled encoding03
• 11:57 zhuyifei1999_: `pip install -U youtube-dl` on all v2c worker instances, version 2017.1.10 -> 2017.2.16

2017-02-07

• 20:04 zhuyifei1999_: git pulling v2c frontend to 74a4ad8 and restarting webservice
• 19:31 zhuyifei1999_: git pulling v2c frontend to a651871 and restarting webservice
• 17:15 zhuyifei1999_: git pulling v2c frontend to bc77044 and restarting webservice
• 16:54 zhuyifei1999_: git pulling v2c frontend to 2e55341 and restarting webservice

2017-02-05

• 10:56 zhuyifei1999_: Git pulling v2c frontend to 9c86a9f and restarting webservice
• 10:52 zhuyifei1999_: `$ pip install -U youtube-dl` from 2017.1.10 to 2017.2.4.1 on v2c frontend

2017-01-29

• 08:53 zhuyifei1999_: git pulling frontend to master (1cad581) and restarting webservice

2017-01-25

• 10:21 zhuyifei1999_: `$ sudo apt update && sudo apt dist-upgrade` on all hosts
• 07:48 zhuyifei1999_: repooling encoding02

2017-01-24

• 21:33 zhuyifei1999_: depooling encoding02 & 03, repooling encoding01
• 21:30 zhuyifei1999_: git pulling v2c code to 8cf788e across all encoding hosts
• 18:27 zhuyifei1999_: repooling encoding02 & encoding03, depooling encoding01 T155455
• 18:23 zhuyifei1999_: killed the rest of tasks on encoding02 & encoding03, they are gonna fail anyways
• 16:17 zhuyifei1999_: encoding03 free disk space < 10G, renicing PID 9136 (50c81be) & 3499 (a32b3fa) & 3611 (0e73d27) to niceness of 1, so these near-complete tasks can finish early and clear some disk space
• 15:32 zhuyifei1999_: git pulling v2c code to b2d6e84 across all encoding hosts T155455
• 14:58 zhuyifei1999_: killing the rest of the tasks on encoding01 because they aren't going to succeed anyways; depooling encoding03; reloading encoding01 T155455
• 14:19 zhuyifei1999_: depooling encoding01 & 02 and waiting for tasks to finish before restarting (reload the code) T155455
• 14:15 zhuyifei1999_: git pulling v2c code to 31479a4 across all encoding hosts T155455
• 14:10 zhuyifei1999_: pip installing pywikibot at 7ac0142 across all encoding hosts T155455

2017-01-23

• 12:10 zhuyifei1999_: starting v2c worker at encoding03 due to flooding

2017-01-20

• 12:01 zhuyifei1999_: restarting all currently running v2c workers (celery{1,2}@encoding0{1,2}) and web frontend due to NFS outage, and we aren't very sure which component is erroring (T155803)

2017-01-15

• 09:28 zhuyifei1999_: pooled encoding02

2017-01-12

• 09:12 zhuyifei1999_: depooling encoding02 + pooling encoding01 + restarting frontend, to upgrade youtube_dl

2017-01-11

• 05:46 zhuyifei1999_: re-enabling v2c frontend & pooling worker at encoding02; transcode queue is < 500 now, and mostly containing relatively small files (T153488)

2016-12-29

• 08:57 zhuyifei1999_: depooled encoding02 per request from operations

2016-12-28

• 10:58 zhuyifei1999_: pooling encoding02, depooling encoding01&03 to update its code

2016-12-27

• 19:20 zhuyifei1999_: depooling encoding02 since the flood is under control
• 15:56 zhuyifei1999_: starting the two other workers because of flood, trying to restore the system to a "ready for new tasks" state

2016-12-22

• 12:10 zhuyifei1999_: depooling v2c workers at encoding02 and encoding03 in order to make transcoding slower and have production videoscalers catch up (T153488)

2016-12-11

• 15:36 zhuyifei1999_: depooling encoding02, less than 5G of disk space left

2016-07-13

• 15:02 andrewbogott: deleting/recreating encoding01

2016-07-11

• 21:22 andrewbogott: deleting instances encoding02 and encoding03, soon to recreate

2016-03-22

• 14:53 andrewbogott: rebooting encoding3 because it's mostly broken

2015-07-31

• 19:39 andrewbogott: migrating encoding01 to a new virt host.

November 19

• 04:57 Ryan_Lane: created video project for development and testing of an external video extension for MediaWiki