You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Revision history of "Nova Resource:Admin/SAL"

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | older 100) (20 | 50 | 100 | 250 | 500)
  • curprev 19:29, 25 March 2020imported>Stashbot 38,291 bytes +285 andrewbogott: dumping a bunch of VMs on cloudvirt1015 to see if it still crashes
  • curprev 19:41, 24 March 2020imported>Stashbot 38,006 bytes +225 jeh: switch cloudvirt1016 from maintenance to standard host aggregate T243327
  • curprev 21:41, 23 March 2020imported>Stashbot 37,781 bytes +305 jeh: restart neutron-l3-agent on cloudnet100[3,4] to pickup policy.yaml changes
  • curprev 14:23, 21 March 2020imported>Stashbot 37,476 bytes +84 andrewbogott: restarting apache2 on labweb1001 and 1002
  • curprev 19:17, 18 March 2020imported>Stashbot 37,392 bytes +334 andrewbogott: deleted a bunch of records from the pdns database on cloudservices1003/1004 which had a record name but the content (where an IP address should be) was NULL, e.g. m.wikidata.beta.wmflabs.org.
  • curprev 17:40, 14 March 2020imported>Stashbot 37,058 bytes +99 jeh: restart maintain-dbusers on labstore1004 T247654
  • curprev 12:39, 13 March 2020imported>Stashbot 36,959 bytes +302 arturo: [codfw1dev] reintroduce address scopes for another round of testing T244851
  • curprev 22:29, 12 March 2020imported>Stashbot 36,657 bytes +130 bstorm_: running puppet across all dumps mounts to make sure active links are shifted to labstore1006
  • curprev 18:38, 11 March 2020imported>Stashbot 36,527 bytes +579 jeh: set icingia downtime until 2020-03-23 on CODFW cloud[control,net,virt] hosts during openstack upgrades
  • curprev 17:02, 10 March 2020imported>Stashbot 35,948 bytes +272 arturo: [codfw1dev] deleting address scopes, bad interaction with our custom NAT setup T247135
  • curprev 18:09, 9 March 2020imported>Stashbot 35,676 bytes +343 arturo: enabling puppet in cloudvirt1006, all services have been restored
  • curprev 14:54, 6 March 2020imported>Stashbot 35,333 bytes +115 andrewbogott: draining all instances off of cloudvirt1006 for T246908
  • curprev 14:24, 5 March 2020imported>Stashbot 35,218 bytes +475 arturo: [codfw1dev] we just enabled BGP session between cloudnet2xxx-dev and cr1-codfw (T245606)
  • curprev 22:22, 4 March 2020imported>Stashbot 34,743 bytes +776 andrewbogott: upgrading designate on cloudservices1003/1004 to Queens
  • curprev 16:54, 2 March 2020imported>Stashbot 33,967 bytes +159 arturo: [codfw1dev] deleted python3-os-ken debian package in cloudnet2003-dev which was installed by hand and had depedency issues
  • curprev 16:32, 29 February 2020imported>Stashbot 33,808 bytes +160 bstorm_: downtimed the smart alert on cloudvirt1009 until Monday since apparently predictive failures flap T244986
  • curprev 22:03, 26 February 2020imported>Stashbot 33,648 bytes +86 jeh: powering down cloudvirt1014 for hardware maintenance
  • curprev 16:08, 25 February 2020imported>Stashbot 33,562 bytes +458 andrewbogott: changing neutron's rabbitmq password because oslo is having trouble parsing some of the characters in the password
  • curprev 12:16, 24 February 2020imported>Stashbot 33,104 bytes +1,060 arturo: [codfw1dev] `root@cloudcontrol2001-dev:~# neutron bgp-speaker-peer-add bgpspeaker cr2-codfw` (T245606)
  • curprev 12:48, 21 February 2020imported>Stashbot 32,044 bytes +771 arturo: [codfw1dev] running `root@cloudcontrol2001-dev:~# neutron bgp-speaker-network-add bgpspeaker wan-transport-codfw` (T245606)
  • curprev 19:22, 20 February 2020imported>Stashbot 31,273 bytes +478 andrewbogott: updating designate pool config for https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/572213/
  • curprev 22:19, 18 February 2020imported>Stashbot 30,795 bytes +429 andrewbogott: transferred the tools.wmcloud.org. to the tools project
  • curprev 10:35, 14 February 2020imported>Stashbot 30,366 bytes +477 arturo: running `root@cloudcontrol2001-dev:~# designate server-create --name ns1.openstack.codfw1dev.wikimediacloud.org.` (T243766)
  • curprev 13:38, 12 February 2020imported>Stashbot 29,889 bytes +270 arturo: [codfw1dev] add reference to subnetpool to the instance subnet `MariaDB [neutron]> update subnets set subnetpool_id='d129650d-d4be-4fe1-b13e-6edb5565cb4a' where id = '7adfcebe-b3d0-4315-92fe-e8365cc80668';` (T244851)
  • curprev 13:46, 11 February 2020imported>Stashbot 29,619 bytes +570 arturo: [codfw1dev] creating some neutron objects to investigate T244851 (subnets, subnet pools, address scopes, ...)
  • curprev 18:11, 7 February 2020imported>Stashbot 29,049 bytes +106 jeh: shutdown cloudvirt1016 for hardware maintenance T241882
  • curprev 14:44, 6 February 2020imported>Stashbot 28,943 bytes +169 jeh: update apt packages on cloudvirt1015 T220853
  • curprev 17:24, 28 January 2020imported>Stashbot 28,774 bytes +1,040 arturo: [codfw1dev] root@cloudcontrol2001-dev:~# designate server-create --name ns0.openstack.codfw1dev.wikimediacloud.org. (T243766)
  • curprev 12:45, 27 January 2020imported>Stashbot 27,734 bytes +495 arturo: [codfw1dev] manually move the new domain to the `cloudinfra-codfw1dev` project clouddb2001-dev: `[designate]> update zones set tenant_id='cloudinfra-codfw1dev' where id = '4c75410017904858a5839de93c9e8b3d';` T243556
  • curprev 15:10, 24 January 2020imported>Stashbot 27,239 bytes +185 jeh: remove icinga downtime for cloudvirt1013 T241313
  • curprev 17:43, 21 January 2020imported>Stashbot 27,054 bytes +253 bstorm_: remounting /mnt/nfs/dumps-labstore1007.wikimedia.org/ on all dumps-mounting projects
  • curprev 16:59, 15 January 2020imported>Stashbot 26,801 bytes +144 bd808: Changed the config for cloud-announce mailing list so that lsit admins do not get bounce unsubscribe notices
  • curprev 14:03, 14 January 2020imported>Stashbot 26,657 bytes +395 arturo: icinga downtime all cloudvirts for another 2h for fixing some icinga checks
  • curprev 13:34, 13 January 2020imported>Stashbot 26,262 bytes +269 arturo: [¢odfw1dev] prevent neutron from allocating floating IPs from the wrong subnet by doing `neutron subnet-update --allocation-pool start=208.80.153.190,end=208.80.153.190 cloud-instances-transport1-b-codfw` (T242594)
  • curprev 13:27, 10 January 2020imported>Stashbot 25,993 bytes +167 arturo: cloudvirt1009: virsh undefine i-000069b6. This is tools-elastic-01 which is running on cloudvirt1008 (so, leaked on cloudvirt1009)
  • curprev 11:12, 9 January 2020imported>Stashbot 25,826 bytes +397 arturo: running `MariaDB [nova_eqiad1]> update quota_usages set in_use='0' where project_id='etytree';` (T242332)
  • curprev 10:53, 8 January 2020imported>Stashbot 25,429 bytes +111 arturo: icinga downtime all cloudvirts for 30 minutes to re-create all canary VMs"
  • curprev 11:12, 7 January 2020imported>Stashbot 25,318 bytes +228 arturo: icinga-downtime everything cloud* for 30 minutes to merge nova scheduler changes
  • curprev 13:45, 6 January 2020imported>Stashbot 25,090 bytes +110 andrewbogott: restarting nova-api and nova-conductor on cloudcontrol1003 and 1004
  • curprev 16:34, 4 January 2020imported>Stashbot 24,980 bytes +130 arturo: icinga downtime cloudvirt1024 for 2 months because hardware errors (T241884)
  • curprev 11:46, 31 December 2019imported>Stashbot 24,850 bytes +158 andrewbogott: I couldn't!
  • curprev 10:13, 25 December 2019imported>Stashbot 24,692 bytes +205 arturo: icinga downtime for 30 minutes the whole cloud* lab* fleet to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/560575 (will restart some openstack components)
  • curprev 15:13, 24 December 2019imported>Stashbot 24,487 bytes +188 arturo: icinga downtime all the lab* fleet for nova password change for 1h
  • curprev 11:13, 23 December 2019imported>Stashbot 24,299 bytes +174 arturo: enable puppet in cloudcontrol1003/1004
  • curprev 23:48, 22 December 2019imported>Stashbot 24,125 bytes +290 andrewbogott: restarting nova-conductor and nova-api on cloudcontrol1003 and 1004
  • curprev 12:43, 20 December 2019imported>Stashbot 23,835 bytes +83 arturo: icinga downtime cloudmetrics1001 for 128 hours
  • curprev 12:55, 18 December 2019imported>Stashbot 23,752 bytes +191 arturo: [codfw1dev] created a new subnet neutron object to hold the new CIDR for floating IPs (cloud-codfw1dev-floating - 185.15.57.0/29) T239347
  • curprev 07:21, 17 December 2019imported>Stashbot 23,561 bytes +85 andrewbogott: deploying horizon/train to labweb1001/1002
  • curprev 06:11, 12 December 2019imported>Stashbot 23,476 bytes +173 arturo: schedule 4h downtime for labstores
  • curprev 06:28, 2 December 2019imported>Stashbot 23,303 bytes +611 andrewbogott: running nova-manage db sync on eqiad1
  • curprev 19:27, 18 November 2019imported>Stashbot 22,692 bytes +897 andrewbogott: repooling labsdb1011
  • curprev 20:04, 15 November 2019imported>Stashbot 21,795 bytes +681 andrewbogott: repool labdb1011 (T237509)
  • curprev 13:10, 11 November 2019imported>Stashbot 21,114 bytes +217 arturo: cloudweb2001-dev: disable puppet and redirect stderr in the loadExitNodes.php cron script to prevent cronspam while we investigate the cause of the issue (T237971)
  • curprev 11:59, 5 November 2019imported>Stashbot 20,897 bytes +171 arturo: icinga downtime for 1h cloudcontrol1004, cloudnet1003, cloudvirt1017/1020/1022 for PDU operations in the rack T227542
  • curprev 21:55, 4 November 2019imported>Stashbot 20,726 bytes +145 andrewbogott: deleting a ton of wikitech hiera pages that were either no-ops or refer to nonexistent VMs or prefixes
  • curprev 11:01, 31 October 2019imported>Stashbot 20,581 bytes +151 arturo: icinga-downtimed cloudvirt1030 and cloudservices1003 for 1h due to PDU upgrade operations T227543
  • curprev 22:43, 30 October 2019imported>Stashbot 20,430 bytes +232 jeh: reboot cloud-bootstrapvz-stretch to resolve bad bootstrapvz build
  • curprev 10:45, 25 October 2019imported>Stashbot 20,198 bytes +176 arturo: icinga downtime toolschecker for 1 to upgrade clouddb1002 mariadb (toolsdb secondary) (T236384 , T236420)
  • curprev 12:30, 24 October 2019imported>Stashbot 20,022 bytes +517 arturo: starting cloudvirt1019, PDU operations ended (T227540)
  • curprev 09:23, 23 October 2019imported>Stashbot 19,505 bytes +293 arturo: cloudvirt1026 reboot ended OK
  • curprev 16:01, 18 October 2019imported>Stashbot 19,212 bytes +420 arturo: created the `eqiad1.wikimedia.cloud` DNS zone (T235846)
  • curprev 21:59, 16 October 2019imported>Stashbot 18,792 bytes +512 jeh: resync wiki replica tool and user accounts T235697
  • curprev 13:30, 15 October 2019imported>Stashbot 18,280 bytes +97 jeh: creating indexes and views for banwiki T234770
  • curprev 18:55, 10 October 2019imported>Stashbot 18,183 bytes +250 bd808: Created indexes and views for nqowiki (T230543)
  • curprev 10:44, 9 October 2019imported>Stashbot 17,933 bytes +531 arturo: cloudvirt1013 rebooted well
  • curprev 14:07, 7 October 2019imported>Stashbot 17,402 bytes +196 arturo: horizon is disabled for maintenance (T212302)
  • curprev 15:23, 2 October 2019imported>Stashbot 17,206 bytes +779 arturo: codfw1dev renaming net/subnet objects to a more modern naming scheme T233665
  • curprev 10:21, 30 September 2019imported>Stashbot 16,427 bytes +290 arturo: we installed ferm in every VM by mistake. Deleting it and forcing a puppet agent run to try to go back to a clean state.
  • curprev 10:39, 18 August 2019imported>Stashbot 16,137 bytes +149 arturo: rebooting cloudvirt1023 for new interface names configuration
  • curprev 17:17, 5 August 2019imported>Stashbot 15,988 bytes +136 bd808: Set downtime on gridengine and kubernetes webservice checks in icinga until 2019-09-02 (flaky tests)
  • curprev 20:14, 29 July 2019imported>Stashbot 15,852 bytes +114 bd808: Restarted maintain-kubeusers on tools-k8s-master-01 (T194859)
  • curprev 12:32, 25 July 2019imported>Stashbot 15,738 bytes +688 arturo: eqiad1/glance: debian-9.9-stretch image deprecates debian-9.8-stretch (T228983)
  • curprev 19:43, 23 July 2019imported>Stashbot 15,050 bytes +98 andrewbogott: restarting rabbitmq-server on cloudcontrol1003 and 1004
  • curprev 23:44, 22 July 2019imported>Stashbot 14,952 bytes +114 bd808: Restarted maintain-kubeusers on tools-k8s-master-01 (T228529)
  • curprev 22:07, 11 July 2019imported>Stashbot 14,838 bytes +343 bd808: Ran `sudo systemctl stop designate_floating_ip_ptr_records_updater.service` on cloudcontrol1003
  • curprev 16:05, 25 June 2019imported>Stashbot 14,495 bytes +213 bstorm_: updated python3.4 to update4 wherever it was installed on Jessie VMs to prevent issues with broken update3.
  • curprev 15:56, 3 June 2019imported>Stashbot 14,282 bytes +7,003 arturo: T221769 rebooting cloudservices1003 after bootstrapping is apparently completed
  • curprev 20:00, 13 January 2019imported>Stashbot 7,279 bytes +441 andrewbogott: VPS proxies are now running in eqiad1 on proxy-01. Old VMs will wait a bit for deletion. T213540
  • curprev 22:21, 9 January 2019imported>Stashbot 6,838 bytes +85 bd808: neutron quota-update --tenant-id tools --port 256
  • curprev 18:59, 8 January 2019imported>Stashbot 6,753 bytes +257 bd808: Definately did NOT delete uid=novaadmin,ou=people,dc=wikimedia,dc=org
  • curprev 22:03, 6 January 2019imported>Stashbot 6,496 bytes +126 bd808: Set floatingip quota of 60 for tools project in eqiad1-r region (T212360)
  • curprev 17:10, 20 December 2018imported>Stashbot 6,370 bytes +100 arturo: T207663 renumbered transport network in eqiad1
  • curprev 17:59, 5 December 2018imported>Stashbot 6,270 bytes +130 arturo: T207663 changed labtestn transport network addressing from private to public
  • curprev 13:25, 3 December 2018imported>Stashbot 6,140 bytes +107 arturo: T202886 create again PTR records after dnsleak.py fix
  • curprev 14:08, 30 November 2018imported>Stashbot 6,033 bytes +126 arturo: running dns leaks cleanup `root@cloudcontrol1003:~# /root/novastats/dnsleaks.py --delete`
  • curprev 17:33, 28 November 2018imported>Stashbot 5,907 bytes +94 gtirloni: deleted contintcloud project (T209644)
  • curprev 13:32, 27 November 2018imported>Stashbot 5,813 bytes +113 gtirloni: enabled DRBD stats collection on labstore100[4-5] T208446
  • curprev 07:12, 22 November 2018imported>Stashbot 5,700 bytes +76 gtirloni: deployed new debian-9.6-stretch image
  • curprev 10:48, 21 November 2018imported>Stashbot 5,624 bytes +134 arturo: re-created compat-net as not shared in labtestn to test stuff related to T209954
  • curprev 12:43, 16 November 2018imported>Stashbot 5,490 bytes +244 gtirloni: armed keyholder on labpuppetmaster1001/1002 after reboots
  • curprev 17:19, 14 November 2018imported>Stashbot 5,246 bytes +869 gtirloni: added cloudvirt1016 to scheduler pool (T209426)
  • curprev 18:17, 9 November 2018imported>Stashbot 4,377 bytes +96 gtirloni: restarted neutron-linuxbridge-agent on cloudvirt1018/1023
  • curprev 11:00, 8 November 2018imported>Stashbot 4,281 bytes +123 gtirloni: Added novaproxy-02 to $CACHES
  • curprev 13:49, 7 November 2018imported>Stashbot 4,158 bytes +144 arturo: T208733 moving labvirt1017 from main deployment to eqiad1 and renaming it to cloudvirt1017
  • curprev 16:24, 22 October 2018imported>Stashbot 4,014 bytes +248 arturo: T206261 another update to dmz_cidr in eqiad1
  • curprev 12:02, 19 October 2018imported>Stashbot 3,766 bytes +392 arturo: revert change in dmz_cidr in eqiad1 for now (T206261)
  • curprev 10:40, 26 September 2018imported>Stashbot 3,374 bytes +753 arturo: T205524 all sorts of restarts in all neutron daemons
  • curprev 22:08, 17 September 2018imported>Stashbot 2,621 bytes +2,384 bd808: Granted gtirloni project roles of admin, projectadmin, and user
  • curprev 00:55, 9 February 2018imported>Stashbot 237 bytes +211 bd808: Added Arturo Borrero Gonzalez and Bstorm as project members
  • curprev 21:02, 8 October 2015imported>Andrew Bogott 26 bytes +26
(newest | oldest) View ( | older 100) (20 | 50 | 100 | 250 | 500)