You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Nova Resource:Codesearch/SAL: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Rush
No edit summary
 
imported>Stashbot
(legoktm: added Krinkle as project admin, Gerrit +2)
 
(69 intermediate revisions by the same user not shown)
Line 1: Line 1:
=== 2022-06-17 ===
* 20:04 legoktm: added Krinkle as project admin, Gerrit +2
=== 2022-05-20 ===
* 05:16 legoktm: restarting to pick up label changes
=== 2022-04-09 ===
* 03:55 legoktm: restarting everything to add new "wmcs" instance
=== 2022-03-15 ===
* 02:09 legoktm: forcing manual run of codesearch-write-config for [[phab:T303641|T303641]]
=== 2021-11-03 ===
* 16:06 legoktm: manually triggered codesearch-write-config job to pick up fix for [[phab:T294915|T294915]] immediately
=== 2021-10-24 ===
* 22:39 legoktm: one more restart of everything, switched hound container over to bullseye for newer git
* 22:20 legoktm: restarted everything to add new prometheus metrics endpoint: https://codesearch.wmcloud.org/_metrics
=== 2021-10-23 ===
* 16:07 majavah: hard reboot codesearch8 after OOM crash
=== 2021-05-21 ===
* 16:08 legoktm: restarting everything for [[phab:T283317|T283317]]
=== 2021-05-20 ===
* 23:45 legoktm: restarting to switch docker images
=== 2021-04-14 ===
* 01:38 legoktm: deleting codesearch6 instance ([[phab:T279763|T279763]])
=== 2021-04-13 ===
* 20:53 legoktm: switched codesearch.wmcloud.org and codesearch-beta.wmcloud.org to point to codesearch8 ([[phab:T279763|T279763]])
=== 2021-04-09 ===
* 08:57 legoktm: deleting codesearch7, creating codesearch8 as the correct size ([[phab:T279763|T279763]])
* 08:52 legoktm: creating codesearch7 instance ([[phab:T279763|T279763]])
* 08:48 legoktm: moved role::labs::lvm::srv from project puppet to codesearch06's puppet with the goal of moving to cinder
=== 2021-03-05 ===
* 23:45 legoktm: restarting for new shouthow profile ([[phab:T253597|T253597]])
=== 2021-01-26 ===
* 02:01 legoktm: restarting for new puppet profile ([[phab:T272947|T272947]])
=== 2021-01-09 ===
* 00:52 legoktm: bunch of restarts, repos with non-master default branches should be properly supported
=== 2020-11-25 ===
* 19:03 legoktm: fixed permissions on /srv/codesearch/.git/objects, forcing puppet run which is triggering restarts
=== 2020-11-18 ===
* 22:45 Amir1: restart codesearch to pick up new config ([[phab:T268187|T268187]])
=== 2020-11-09 ===
* 11:05 Amir1: restart codesearch
=== 2020-10-30 ===
* 22:43 GarbageMonster: restarting codesearch to pick up new config ([[phab:T266909|T266909]])
=== 2020-10-05 ===
* 04:57 legoktm: restarting to deploy [[phab:T264557|T264557]]
=== 2020-09-22 ===
* 20:33 legoktm: (re)starting all hound instances now that gerrit-replica is back up
=== 2020-08-28 ===
* 19:17 Amir1: restarting codesearch to pick up the new config ([[phab:T261517|T261517]])
=== 2020-08-12 ===
* 08:57 legoktm: restarting proxy to implement etag changes
=== 2020-08-11 ===
* 23:48 legoktm: restarting hound-deployed for the same reason
* 21:30 Amir1: restarting hound-search to pick up the new config for services ([[phab:T260124|T260124]])
=== 2020-07-26 ===
* 05:56 legoktm: restarting everything (again) for https://gerrit.wikimedia.org/r/616317
* 04:50 legoktm: restarting everything to deploy https://gerrit.wikimedia.org/r/616316
=== 2020-07-24 ===
* 13:39 Amir1: restarting codesearch to pick up the new config ([[phab:T231572|T231572]])
=== 2020-07-22 ===
* 14:03 Amir1: restart codesearch to pick up new config (adding mediawiki/vagrant)
=== 2020-07-13 ===
* 21:04 Amir1: create codesearch.wmcloud.org dns proxy ([[phab:T256206|T256206]])
=== 2020-06-28 ===
* 09:31 legoktm: root@codesearch6:~# chown codesearch:codesearch -R /srv/codesearch
=== 2020-06-11 ===
* 11:04 legoktm: restarting everything after gerrit-replica 502s fixed [[phab:T255094|T255094]] [[phab:T255125|T255125]]
=== 2020-03-06 ===
* 18:03 bd808: Forced Puppet run on codesearch6 to apply Puppet fix for [[phab:T246017|T246017]]
=== 2020-02-25 ===
* 21:52 Amir1: restart hound proxy
* 21:50 Amir1: sudo service hound-libraries restart ([[phab:T246120|T246120]] [[phab:T246121|T246121]])
=== 2020-02-24 ===
* 17:51 bd808: `systemctl restart docker` on codesearch6 ([[phab:T246017|T246017]])
* 17:37 bd808: `systemctl restart hound-search.service` on codesearch6 ([[phab:T246017|T246017]])
=== 2020-02-22 ===
* 16:54 Amir1: hard reboot of codesearch6 ([[phab:T245920|T245920]])
=== 2020-02-19 ===
* 14:48 Amir1: restarting hound_proxy to pick up the new changes
=== 2020-02-13 ===
* 19:02 Reedy: is down "Hound is still starting up, please wait a few minutes for the initial indexing to complete."
=== 2020-01-25 ===
* 04:14 legoktm: restarted for new libraries search profile
=== 2020-01-23 ===
* 02:00 legoktm: shutting down and deleting temporary codesearch5 instance
* 01:59 legoktm: shutting down old codesearch4 instance (but not deleting yet)
* 01:57 legoktm: deleting temporary codesearch6.wmflabs.org domain
* 01:56 legoktm: retargetting codesearch.wmflabs.org to codesearch6 ([[phab:T242319|T242319]])
=== 2020-01-19 ===
* 06:08 legoktm: temporarily taking everything down to free up disk space ([[phab:T243121|T243121]])
=== 2020-01-17 ===
* 07:37 legoktm: rolling restart of hound instances to test new bootstrapping overload protection
=== 2020-01-16 ===
* 20:17 legoktm: legoktm@codesearch5:~$ sudo crontab -u codesearch -r
=== 2020-01-11 ===
* 03:57 legoktm: added mutante as project admin to help with puppetization efforts
=== 2019-12-22 ===
* 12:29 Amir1: ladsgroup@codesearch4:~$ sudo service hound-armchairgm restart ([[phab:T241319|T241319]])
=== 2019-12-15 ===
* 01:30 legoktm: rolling restart of all hound instances because of gerrit-replica outage ([[phab:T240776|T240776]], [[phab:T240763|T240763]])
* 00:40 Amir1: sudo service hound-search restart ([[phab:T240776|T240776]])
=== 2019-10-16 ===
* 10:17 Amir1: ladsgroup@codesearch4:~$ sudo service hound-services restart ([[phab:T235596|T235596]])
=== 2019-09-30 ===
* 16:14 Amir1: soft-rebooting codesearch4
* 15:49 Amir1: ladsgroup@codesearch4:~$ sudo MODE=restart /srv/codesearch/manage.sh ([[phab:T234211|T234211]])
=== 2019-09-29 ===
* 21:49 Amir1: git pull for puppet ([[phab:T219248|T219248]])
=== 2019-09-18 ===
* 00:06 Amir1: another restart
=== 2019-09-17 ===
* 23:48 Amir1: restart to pick up the shiny new "services" section
=== 2019-09-16 ===
* 15:56 Amir1: ladsgroup@codesearch4:~$ sudo MODE=restart /srv/codesearch/manage.sh (It seems it's down, forcing a restart)
=== 2019-09-03 ===
* 22:39 Amir1: "sudo MODE=restart /srv/codesearch/manage.sh"
=== 2019-08-07 ===
* 20:10 legoktm: deleting now unneeded ggmirror-02 instance
* 20:04 legoktm: switched over to using gerrit-replica
=== 2019-07-17 ===
* 00:53 legoktm: manual rolling restart of everything to start using new gerrit mirror
=== 2019-07-16 ===
* 23:04 legoktm: deleted medium sized ggmirror-01 instance, it ran out of disk space. recreated as ggmirror-02 with size large
=== 2019-07-11 ===
* 15:01 legoktm: legoktm@codesearch4:/srv/puppet$ git pull
=== 2019-06-06 ===
* 18:15 legoktm: force updated config and doing a rolling restart ([[phab:T223771|T223771]])
=== 2019-05-24 ===
* 21:54 legoktm: maintenance to reduce disk space
=== 2019-03-28 ===
* 08:13 legoktm: legoktm@codesearch4:/srv/puppet$ git pull # [[phab:T219248|T219248]]
=== 2019-01-03 ===
* 00:35 legoktm: restarting hound instances for [[phab:T211999|T211999]]
=== 2018-12-03 ===
* 06:17 legoktm: deleted codesearch3 instance ([[phab:T210372|T210372]])
=== 2018-11-26 ===
* 18:33 duh: codesearch4: deleted all of /srv/hound/hound-core/data/ and restarted the hound-core service for a fresh clone ([[phab:T210409|T210409]])
* 06:36 legoktm: retarget codesearch.wmflabs.org to codesearch4 instance
* 06:14 legoktm: created and setting up codesearch4 image based on Debian Stretch
* 06:03 legoktm: deleted codesearch-sourcegraph1 instance
=== 2018-11-16 ===
* 13:51 andrewbogott: migrating project to eqiad1
=== 2018-11-05 ===
* 07:46 legoktm: Implemented automatic restarts of hound to regularly pick up new config changes
=== 2018-09-27 ===
* 23:35 legoktm: restarting hound for config change (https://gerrit.wikimedia.org/r/463390)
=== 2018-09-21 ===
* 07:36 legoktm: restarted to pick up new config / repository lists
=== 2018-09-05 ===
* 05:25 legoktm: restarted all hound backends after deleting all of their data
* 05:16 legoktm: stopping hound  backends to clean up disk space
* 04:52 legoktm: restarted hound for [[gerrit:458105]]
=== 2018-08-17 ===
* 18:17 legoktm: rolling restart to pick up new config
=== 2018-07-09 ===
* 13:07 legoktm: restarting everything for config update
=== 2018-04-13 ===
* 17:41 legoktm: booted up codesearch3, it was down for some reason
=== 2018-03-02 ===
* 02:12 legoktm: deleted codesearch2 instance
* 02:09 legoktm: pointed codesearch.wmflabs.org at codesearch3:3002
* 02:05 legoktm: ran `mkfs.ext4 -T news /dev/mapper/vd-second--local--disk` (h/t bd808) to quintuple the number of inodes
* 01:52 legoktm: applied role::labs::lvm::srv role to codesearch3
* 01:52 legoktm: created codesearch3 instance (jessie, medium flavor)
=== 2018-02-28 ===
* 04:44 legoktm: restarted hound-search service ([[phab:T188477|T188477]]), also I restarted hound-extensions earlier ([[phab:T188450|T188450]])
=== 2018-02-24 ===
* 08:47 legoktm: added MaxSem as a project admin
=== 2018-02-23 ===
* 22:23 legoktm: added d3r1ck's correct account as project admin
=== 2018-02-14 ===
* 20:19 legoktm: added Addshore, D3r1ck01, Ladsgroup, and Prtksxna to Gerrit labs-codesearch project
* 20:12 legoktm: added Addshore, D3r1ck01, Ladsgroup, and Prtksxna as project admins of codesearch
=== 2018-01-18 ===
* 13:32 legoktm: rebooting codesearch2 to see how well it recovers
* 11:36 legoktm: legoktm@codesearch2:~$ sudo systemctl enable hound_proxy
=== 2018-01-16 ===
* 22:49 chasemp: restarted everything I could find in the stack :)
=== 2017-12-21 ===
* 23:28 legoktm: restarting hound instances to pick up config change
{{SAL|Project Name=codesearch}}
{{SAL|Project Name=codesearch}}
<noinclude>[[Category:SAL]]</noinclude>

Latest revision as of 20:04, 17 June 2022

2022-06-17

  • 20:04 legoktm: added Krinkle as project admin, Gerrit +2

2022-05-20

  • 05:16 legoktm: restarting to pick up label changes

2022-04-09

  • 03:55 legoktm: restarting everything to add new "wmcs" instance

2022-03-15

  • 02:09 legoktm: forcing manual run of codesearch-write-config for T303641

2021-11-03

  • 16:06 legoktm: manually triggered codesearch-write-config job to pick up fix for T294915 immediately

2021-10-24

  • 22:39 legoktm: one more restart of everything, switched hound container over to bullseye for newer git
  • 22:20 legoktm: restarted everything to add new prometheus metrics endpoint: https://codesearch.wmcloud.org/_metrics

2021-10-23

  • 16:07 majavah: hard reboot codesearch8 after OOM crash

2021-05-21

  • 16:08 legoktm: restarting everything for T283317

2021-05-20

  • 23:45 legoktm: restarting to switch docker images

2021-04-14

  • 01:38 legoktm: deleting codesearch6 instance (T279763)

2021-04-13

  • 20:53 legoktm: switched codesearch.wmcloud.org and codesearch-beta.wmcloud.org to point to codesearch8 (T279763)

2021-04-09

  • 08:57 legoktm: deleting codesearch7, creating codesearch8 as the correct size (T279763)
  • 08:52 legoktm: creating codesearch7 instance (T279763)
  • 08:48 legoktm: moved role::labs::lvm::srv from project puppet to codesearch06's puppet with the goal of moving to cinder

2021-03-05

  • 23:45 legoktm: restarting for new shouthow profile (T253597)

2021-01-26

  • 02:01 legoktm: restarting for new puppet profile (T272947)

2021-01-09

  • 00:52 legoktm: bunch of restarts, repos with non-master default branches should be properly supported

2020-11-25

  • 19:03 legoktm: fixed permissions on /srv/codesearch/.git/objects, forcing puppet run which is triggering restarts

2020-11-18

  • 22:45 Amir1: restart codesearch to pick up new config (T268187)

2020-11-09

  • 11:05 Amir1: restart codesearch

2020-10-30

  • 22:43 GarbageMonster: restarting codesearch to pick up new config (T266909)

2020-10-05

  • 04:57 legoktm: restarting to deploy T264557

2020-09-22

  • 20:33 legoktm: (re)starting all hound instances now that gerrit-replica is back up

2020-08-28

  • 19:17 Amir1: restarting codesearch to pick up the new config (T261517)

2020-08-12

  • 08:57 legoktm: restarting proxy to implement etag changes

2020-08-11

  • 23:48 legoktm: restarting hound-deployed for the same reason
  • 21:30 Amir1: restarting hound-search to pick up the new config for services (T260124)

2020-07-26

2020-07-24

  • 13:39 Amir1: restarting codesearch to pick up the new config (T231572)

2020-07-22

  • 14:03 Amir1: restart codesearch to pick up new config (adding mediawiki/vagrant)

2020-07-13

  • 21:04 Amir1: create codesearch.wmcloud.org dns proxy (T256206)

2020-06-28

  • 09:31 legoktm: root@codesearch6:~# chown codesearch:codesearch -R /srv/codesearch

2020-06-11

  • 11:04 legoktm: restarting everything after gerrit-replica 502s fixed T255094 T255125

2020-03-06

  • 18:03 bd808: Forced Puppet run on codesearch6 to apply Puppet fix for T246017

2020-02-25

  • 21:52 Amir1: restart hound proxy
  • 21:50 Amir1: sudo service hound-libraries restart (T246120 T246121)

2020-02-24

  • 17:51 bd808: `systemctl restart docker` on codesearch6 (T246017)
  • 17:37 bd808: `systemctl restart hound-search.service` on codesearch6 (T246017)

2020-02-22

  • 16:54 Amir1: hard reboot of codesearch6 (T245920)

2020-02-19

  • 14:48 Amir1: restarting hound_proxy to pick up the new changes

2020-02-13

  • 19:02 Reedy: is down "Hound is still starting up, please wait a few minutes for the initial indexing to complete."

2020-01-25

  • 04:14 legoktm: restarted for new libraries search profile

2020-01-23

  • 02:00 legoktm: shutting down and deleting temporary codesearch5 instance
  • 01:59 legoktm: shutting down old codesearch4 instance (but not deleting yet)
  • 01:57 legoktm: deleting temporary codesearch6.wmflabs.org domain
  • 01:56 legoktm: retargetting codesearch.wmflabs.org to codesearch6 (T242319)

2020-01-19

  • 06:08 legoktm: temporarily taking everything down to free up disk space (T243121)

2020-01-17

  • 07:37 legoktm: rolling restart of hound instances to test new bootstrapping overload protection

2020-01-16

  • 20:17 legoktm: legoktm@codesearch5:~$ sudo crontab -u codesearch -r

2020-01-11

  • 03:57 legoktm: added mutante as project admin to help with puppetization efforts

2019-12-22

  • 12:29 Amir1: ladsgroup@codesearch4:~$ sudo service hound-armchairgm restart (T241319)

2019-12-15

  • 01:30 legoktm: rolling restart of all hound instances because of gerrit-replica outage (T240776, T240763)
  • 00:40 Amir1: sudo service hound-search restart (T240776)

2019-10-16

  • 10:17 Amir1: ladsgroup@codesearch4:~$ sudo service hound-services restart (T235596)

2019-09-30

  • 16:14 Amir1: soft-rebooting codesearch4
  • 15:49 Amir1: ladsgroup@codesearch4:~$ sudo MODE=restart /srv/codesearch/manage.sh (T234211)

2019-09-29

  • 21:49 Amir1: git pull for puppet (T219248)

2019-09-18

  • 00:06 Amir1: another restart

2019-09-17

  • 23:48 Amir1: restart to pick up the shiny new "services" section

2019-09-16

  • 15:56 Amir1: ladsgroup@codesearch4:~$ sudo MODE=restart /srv/codesearch/manage.sh (It seems it's down, forcing a restart)

2019-09-03

  • 22:39 Amir1: "sudo MODE=restart /srv/codesearch/manage.sh"

2019-08-07

  • 20:10 legoktm: deleting now unneeded ggmirror-02 instance
  • 20:04 legoktm: switched over to using gerrit-replica

2019-07-17

  • 00:53 legoktm: manual rolling restart of everything to start using new gerrit mirror

2019-07-16

  • 23:04 legoktm: deleted medium sized ggmirror-01 instance, it ran out of disk space. recreated as ggmirror-02 with size large

2019-07-11

  • 15:01 legoktm: legoktm@codesearch4:/srv/puppet$ git pull

2019-06-06

  • 18:15 legoktm: force updated config and doing a rolling restart (T223771)

2019-05-24

  • 21:54 legoktm: maintenance to reduce disk space

2019-03-28

  • 08:13 legoktm: legoktm@codesearch4:/srv/puppet$ git pull # T219248

2019-01-03

  • 00:35 legoktm: restarting hound instances for T211999

2018-12-03

  • 06:17 legoktm: deleted codesearch3 instance (T210372)

2018-11-26

  • 18:33 duh: codesearch4: deleted all of /srv/hound/hound-core/data/ and restarted the hound-core service for a fresh clone (T210409)
  • 06:36 legoktm: retarget codesearch.wmflabs.org to codesearch4 instance
  • 06:14 legoktm: created and setting up codesearch4 image based on Debian Stretch
  • 06:03 legoktm: deleted codesearch-sourcegraph1 instance

2018-11-16

  • 13:51 andrewbogott: migrating project to eqiad1

2018-11-05

  • 07:46 legoktm: Implemented automatic restarts of hound to regularly pick up new config changes

2018-09-27

2018-09-21

  • 07:36 legoktm: restarted to pick up new config / repository lists

2018-09-05

  • 05:25 legoktm: restarted all hound backends after deleting all of their data
  • 05:16 legoktm: stopping hound backends to clean up disk space
  • 04:52 legoktm: restarted hound for gerrit:458105

2018-08-17

  • 18:17 legoktm: rolling restart to pick up new config

2018-07-09

  • 13:07 legoktm: restarting everything for config update

2018-04-13

  • 17:41 legoktm: booted up codesearch3, it was down for some reason

2018-03-02

  • 02:12 legoktm: deleted codesearch2 instance
  • 02:09 legoktm: pointed codesearch.wmflabs.org at codesearch3:3002
  • 02:05 legoktm: ran `mkfs.ext4 -T news /dev/mapper/vd-second--local--disk` (h/t bd808) to quintuple the number of inodes
  • 01:52 legoktm: applied role::labs::lvm::srv role to codesearch3
  • 01:52 legoktm: created codesearch3 instance (jessie, medium flavor)

2018-02-28

  • 04:44 legoktm: restarted hound-search service (T188477), also I restarted hound-extensions earlier (T188450)

2018-02-24

  • 08:47 legoktm: added MaxSem as a project admin

2018-02-23

  • 22:23 legoktm: added d3r1ck's correct account as project admin

2018-02-14

  • 20:19 legoktm: added Addshore, D3r1ck01, Ladsgroup, and Prtksxna to Gerrit labs-codesearch project
  • 20:12 legoktm: added Addshore, D3r1ck01, Ladsgroup, and Prtksxna as project admins of codesearch

2018-01-18

  • 13:32 legoktm: rebooting codesearch2 to see how well it recovers
  • 11:36 legoktm: legoktm@codesearch2:~$ sudo systemctl enable hound_proxy

2018-01-16

  • 22:49 chasemp: restarted everything I could find in the stack :)

2017-12-21

  • 23:28 legoktm: restarting hound instances to pick up config change