You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Nova Resource:Admin/SAL: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

17 March 2021

  • curprev 17:2817:28, 17 March 2021‎ imported>Stashbot‎ 100,320 bytes +432‎ bstorm: restarted the backup-glance-images job to clear errors in systemd T271782

10 March 2021

9 March 2021

5 March 2021

  • curprev 21:4021:40, 5 March 2021‎ imported>Stashbot‎ 97,855 bytes +748‎ andrewbogott: replacing 'observer' role with 'reader' role in eqiad1 T276018

4 March 2021

  • curprev 18:3618:36, 4 March 2021‎ imported>Stashbot‎ 97,107 bytes +1,044‎ andrewbogott: rebooting cloudmetrics1002; the console is hanging

3 March 2021

  • curprev 17:1617:16, 3 March 2021‎ imported>Stashbot‎ 96,063 bytes +1,804‎ andrewbogott: restarting rabbitmq-server on cloudcontrol1003,1004,1005; trying to explain amqp errors in scheduler logs

2 March 2021

  • curprev 17:1617:16, 2 March 2021‎ imported>Stashbot‎ 94,259 bytes +717‎ andrewbogott: rebooting cloudvirt1039 to see if I can trigger T276208

1 March 2021

  • curprev 20:1220:12, 1 March 2021‎ imported>Stashbot‎ 93,542 bytes +347‎ andrewbogott: removing novaadmin from all projects save 'admin' for T274385

28 February 2021

  • curprev 04:5404:54, 28 February 2021‎ imported>Stashbot‎ 93,195 bytes +162‎ andrewbogott: restarted redis-server on tools-redis-1003 and tools-redis-1004 in an attempt to reduce replag, no real change detected

27 February 2021

  • curprev 00:3300:33, 27 February 2021‎ imported>Stashbot‎ 93,033 bytes +4,713‎ andrewbogott: sudo cumin --timeout 500 "A:all and not O{project:clouddb-services}" 'lsb_release -c | grep -i buster && uname -r | grep -v 4.19.0-14-amd64 && reboot'

25 February 2021

  • curprev 14:5614:56, 25 February 2021‎ imported>Stashbot‎ 88,320 bytes +121‎ arturo: deployed wmcs-netns-events daemon to all cloudnet servers (T275483)

24 February 2021

  • curprev 11:0711:07, 24 February 2021‎ imported>Stashbot‎ 88,199 bytes +112‎ arturo: force-reboot cloudmetrics1002, add icinga downtime for 2 hours. Investigating some server issue
  • curprev 00:1700:17, 24 February 2021‎ imported>Stashbot‎ 88,087 bytes +717‎ bstorm: set --property hw_scsi_model=virtio-scsi and --property hw_disk_bus=scsi on the main stretch image in glance on eqiad1 T275430

22 February 2021

  • curprev 17:1517:15, 22 February 2021‎ imported>Stashbot‎ 87,370 bytes +368‎ bstorm: restarting nova-compute on cloudvirt1016 and cloudvirt1036 in case it helps T275411

18 February 2021

17 February 2021

  • curprev 15:5815:58, 17 February 2021‎ imported>Stashbot‎ 86,576 bytes +153‎ arturo: deploying https://gerrit.wikimedia.org/r/c/operations/puppet/+/664845 to cloudnet servers (T268335)

15 February 2021

  • curprev 16:2516:25, 15 February 2021‎ imported>Stashbot‎ 86,423 bytes +395‎ arturo: [codfw1dev] rebooting all cloudgw200x-dev / cloudnet200x-dev servers (T272963)

11 February 2021

  • curprev 12:0112:01, 11 February 2021‎ imported>Stashbot‎ 86,028 bytes +692‎ arturo: [codfw1dev] drop instance `tools-codfw1dev-bastion-1` in `tools-codfw1dev` (was buster, cannot use it yet)

9 February 2021

  • curprev 15:2315:23, 9 February 2021‎ imported>Stashbot‎ 85,336 bytes +224‎ arturo: icinga-downtime for 2h everything *labs *cloud for openstack upgrades

8 February 2021

5 February 2021

  • curprev 10:5910:59, 5 February 2021‎ imported>Stashbot‎ 84,859 bytes +334‎ arturo: icinga-downtime labstore1004 tools share space check for 1 week (T272247)

4 February 2021

  • curprev 10:1210:12, 4 February 2021‎ imported>Stashbot‎ 84,525 bytes +147‎ dcaro: Increasing the memory limit of osds in eqiad from 8589934592(8G) to 12884901888(12G) (T273851)

3 February 2021

  • curprev 09:5909:59, 3 February 2021‎ imported>Stashbot‎ 84,378 bytes +203‎ dcaro: Doing a full vm backup on cloudvirt1024 with the new script (T260692)

2 February 2021

29 January 2021

  • curprev 15:3615:36, 29 January 2021‎ imported>Stashbot‎ 83,829 bytes +155‎ andrewbogott: disabling puppet and some services on eqiad1 cloudcontrol nodes; replacing nova-placement-api with placement-api

28 January 2021

  • curprev 19:4419:44, 28 January 2021‎ imported>Stashbot‎ 83,674 bytes +158‎ andrewbogott: shutting down cloudcontrol2001-dev because it's in a partially upgraded state; will revive when it's time for Train

27 January 2021

22 January 2021

  • curprev 16:4416:44, 22 January 2021‎ imported>Stashbot‎ 83,415 bytes +191‎ andrewbogott: upgrading designate on cloudvirt1003/1004 to OpenStack 'train'

21 January 2021

  • curprev 11:3511:35, 21 January 2021‎ imported>Stashbot‎ 83,224 bytes +338‎ arturo: merging core router firewall changes https://gerrit.wikimedia.org/r/c/operations/homer/public/+/657439 (T209082)

20 January 2021

  • curprev 10:4910:49, 20 January 2021‎ imported>Stashbot‎ 82,886 bytes +1,118‎ arturo: merging core router firewall change https://gerrit.wikimedia.org/r/c/operations/homer/public/+/657302 (T209082)

19 January 2021

18 January 2021

  • curprev 16:0016:00, 18 January 2021‎ imported>Stashbot‎ 81,665 bytes +865‎ dcaro: Codfw1 ceph cluster uprgaded, will wait until tomorrow to see if there's any instability, but everything looks fine (T272303)

17 January 2021

  • curprev 16:5316:53, 17 January 2021‎ imported>Stashbot‎ 80,800 bytes +126‎ arturo: icinga downtime labstore1004 /srv/tools space check for 3 days (T272247)

15 January 2021

  • curprev 13:4113:41, 15 January 2021‎ imported>Stashbot‎ 80,674 bytes +405‎ arturo: icinga downtime labstore1004 maintain-dbuser alert until 2021-01-19 (T272125)

13 January 2021

  • curprev 17:0317:03, 13 January 2021‎ imported>Stashbot‎ 80,269 bytes +927‎ arturo: remove cloudvirt1013 cloudvirt1032 cloudvirt1037 to the 'toobusy' host aggregate to prevent further CPU oversubscribing

12 January 2021

11 January 2021

  • curprev 10:2210:22, 11 January 2021‎ imported>Stashbot‎ 79,167 bytes +573‎ arturo: doubling size of conntrack table in cloudnet servers https://gerrit.wikimedia.org/r/c/operations/puppet/+/655407 (T271058)

10 January 2021

  • curprev 16:0216:02, 10 January 2021‎ imported>Stashbot‎ 78,594 bytes +198‎ andrewbogott: restarting rabbitmq-server on all eqiad1 cloudcontrols

8 January 2021

  • curprev 11:2511:25, 8 January 2021‎ imported>Stashbot‎ 78,396 bytes +1,559‎ arturo: rebooting both cloudnet2002-dev/cloudnet2003-dev to make sure interfaces are set up correctl (T271517)

7 January 2021

  • curprev 15:1915:19, 7 January 2021‎ imported>Stashbot‎ 76,837 bytes +447‎ dcaro: Finished speed tests on cloudcephosd2001-dev, reprovisioning the osd.0 sdc (T271417)

5 January 2021

  • curprev 10:4010:40, 5 January 2021‎ imported>Stashbot‎ 76,390 bytes +134‎ dcaro: removing dumps-[1..*] backups from cloudvirt1024 as they are not needed (T271094)

3 January 2021

  • curprev 07:0607:06, 3 January 2021‎ imported>Stashbot‎ 76,256 bytes +117‎ dcaro: Got a network hiccup on cloudnet1004, keeping track here T271058

28 December 2020

  • curprev 12:3212:32, 28 December 2020‎ imported>Stashbot‎ 76,139 bytes +567‎ arturo: stop doing backups for the dumps project https://gerrit.wikimedia.org/r/c/operations/puppet/+/652182 (T260692)

23 December 2020

  • curprev 15:3815:38, 23 December 2020‎ imported>Stashbot‎ 75,572 bytes +333‎ andrewbogott: restarting rabbitmq on cloudcontrol1004; suspected leaks

22 December 2020

  • curprev 15:3015:30, 22 December 2020‎ imported>Stashbot‎ 75,239 bytes +231‎ dcaro: cleaning up 6778 dangling snapshots for glance images in eqiad (T270478)

19 December 2020

  • curprev 16:1816:18, 19 December 2020‎ imported>Stashbot‎ 75,008 bytes +84‎ dcaro: gzipped a bunch of logs on cloudvirt1004 due to / being out of space
  • curprev 00:1400:14, 19 December 2020‎ imported>Stashbot‎ 74,924 bytes +2,096‎ bstorm: truncated /var/log/debug.1 on cloudcontrol1003 which appears to be the exact same content as the user.log files anyway

17 December 2020

  • curprev 22:1722:17, 17 December 2020‎ imported>Stashbot‎ 72,828 bytes +570‎ andrewbogott: correction to above, set the pg and pgp to 1024 for eqiad1-glance-images

16 December 2020

  • curprev 09:3109:31, 16 December 2020‎ imported>Stashbot‎ 72,258 bytes +121‎ dcaro: removing invalid backups from cloudvirt1024 (196 in total) (T269419)

14 December 2020

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)
Retrieved from "https://wikitech-static.wikimedia.org/wiki/Nova_Resource:Admin/SAL"