Nova Resource:Toolsbeta/SAL

2018-03-05

• 19:33 chicocvenancio: added Zhuyifei1999 as project admin

2018-02-09

• 01:11 bd808: Removed Yuvipanda at user request (T186289)

2017-08-07

• 14:09 andrewbogott: deleted etcd-k8s-CTEST and k8s-master-CTEST

2017-04-26

• 15:38 madhuvishy: add Madhuvishy as projectadmin

2016-10-07

• 19:30 valhallasw`cloud: (puppet certs, to be precise)
• 19:30 valhallasw`cloud: fixed certs on toolsbeta-vagrant3-scfc.toolsbeta.eqiad.wmflabs

2016-10-04

• 19:31 valhallasw`cloud: puppet is broken due to incorrect certificates. Cleaning up ('puppet cert clean toolsbeta-webgrid-lighttpd-1406.toolsbeta.eqiad.wmflabs' on puppetmaster3, 'rm -f /var/lib/puppet/client/ssl/certs/toolsbeta-webgrid-lighttpd-1406.toolsbeta.eqiad.wmflabs.pem' on host, for all hosts that I got emails for)

2016-09-08

• 17:11 bd808: Added BryanDavis (self) to project as admin

2016-08-29

• 19:20 yuvipanda: reboot toolsbeta-master, seems, uh, stuck
• 19:18 yuvipanda: reboot toolsbeta-mail, seems, uh, stuck
• 18:48 yuvipanda: reboot toolsbeta-puppetmaster3, puppet run process became Zommmmbiiiieeee, ate all my brains

2016-07-03

• 15:02 yuvipanda: migrating toolsbeta-valhallasw-puppet-compiler to labvirt1011 to ease pressure on labvirt1010

2016-05-27

• 18:57 valhallasw`cloud: sudo qconf -Ae /var/lib/gridengine/etc/exechosts/toolsbeta-exec-1209.toolsbeta.eqiad.wmflabs

2016-05-26

• 15:08 valhallasw`cloud: toolsbeta-mail has high load (1.0) without clear origin, so rebooting the host

2015-10-13

• 19:21 valhallasw`cloud: started building toolsbeta-bastion.

2015-09-07

• 18:50 valhallasw`cloud: role::bastion is now applied on -exec-101. Now for the package_builder manifest...
• 18:30 valhallasw`cloud: applied role::toollabs::bastion on toolsbeta-exec-101 (spinning up a whole new instance will take ages)

July 4

• 12:57 valhallasw`cloud: restarting toolsbeta-webproxy, no response on port 22

July 2

• 14:55 valhallasw`cloud: toolsbeta-webproxy does not respond at all to SSH; rebooting

July 1

• 19:47 valhallasw`cloud: still can't login :/ not sure if this is a remainder of the NFS failure or something else; maybe a puppet run will solve it?
• 19:44 valhallasw`cloud: restarting toolsbeta-exec-01 and toolsbeta-mail as I can't login

June 7

• 14:44 valhallasw: updated /var/lib/git/operations/puppet to make sure the other hosts get the memo
• 14:42 YuviPanda: run sudo sed -i 's/GlobalSign_CA.pem/ca-certificates.crt/' /etc/ldap/ldap.conf on toolsbeta-puppetmaster3 to fix broken LDAP TLS config

May 11

• 18:14 valhallasw: building toolsbeta-pbuilder to experiment with pbuilder for building packages

May 2

• 11:11 valhallasw`cloud: commenting out include ::elasticsearch::ganglia in role::logstash seems to work. I think we have to write our own tools logstash roles anyway in the end, as the role::logstash code contains e.g. mediawiki specific code
• 10:37 valhallasw`cloud: that doesn't seem to be applied... setting has_ganglia: false manually in wikitech hiera
• 10:30 valhallasw`cloud: pulled new changes into puppetmaster to get https://github.com/wikimedia/operations-puppet/commit/4afd23d8e2905a84ef211ad92e8314173eb743ba in
• 10:25 valhallasw`cloud: set Hiera variable "elasticsearch::cluster_name": toolsbeta-logstash-eqiad
• 10:09 valhallasw`cloud: created toolsbeta-logstash to play around with logstash and figure out what we need for tools (phab:T97861)

April 26

• 18:18 valhallasw`cloud: having some issues with puppet-test, so postponing for now
• 17:12 valhallasw`cloud: deploying https://gerrit.wikimedia.org/r/#/c/206118/ on tools-beta using puppet-test

March 31

• 00:27 andrewbogott: shut down toolsbeta-webgrid-03 to conserve resources. It can be restarted when needed.

September 20

• 20:09 andrewbogott_afk: moved toolsbeta-exec-01 and toolsbeta-scfc-icinga-test off of virt1006

July 22

• 11:36 scfc_de: Removed andrewbogott_afk, Coren, petan, YuviPanda from service group admin to prevent further spamming :-)

August 19

• 12:44 petan: rebooting apache it seems to be frozen

August 4

• 23:50 scfc_de: Added scfc_de to local-admin so I don't log myself out again :-)

July 6

• 19:42 petan: rebooting login

June 26

• 08:03 wm-bot: petrb: updating logsplitter

June 24

• 14:47 wm-bot: petrb: rebooting exec-01 to fix the grid weird info
• 13:43 scfc_de: Made scfc root.
• 13:42 scfc_de: Created toolsbeta-puppetmaster.
• 11:09 YuviPanda: Granted yuvipanda root on toolsbeta

June 21

• 13:46 wm-bot: petrb: rebooting all servers

June 17

• 08:31 petan: switching all instances to nfs

June 16

• 15:37 petan: importing sudo policies of tools
• 15:36 petan: importing security groups of tools
• 15:36 petan: blah