You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org
Portal:Cloud VPS/Infrastructure: Difference between revisions
imported>BryanDavis (Update some things for rebranding) |
imported>Framawiki |
||
| Line 1: | Line 1: | ||
'''Cloud VPS''' is a virtualization cloud that uses [http://www.openstack.org/software/openstack-compute OpenStack Compute]. Base images are managed with [http://docs.openstack.org/developer/glance/ Glance] and authentication uses LDAP-backed [http://docs.openstack.org/developer/keystone/ Keystone]. | '''Cloud VPS''' is a virtualization cloud that uses [http://www.openstack.org/software/openstack-compute OpenStack Compute]. Base images are managed with [http://docs.openstack.org/developer/glance/ Glance] and authentication uses LDAP-backed [http://docs.openstack.org/developer/keystone/ Keystone]. | ||
Cloud VPS currently runs in a single datacenter in Ashburn, Virginia. In the future it will span two or more datacenters, with a slightly different configuration in each. | Cloud VPS currently runs in a single datacenter in Ashburn, Virginia. In the future it will span two or more datacenters, with a slightly different configuration in each. | ||
For troubleshooting immediate | For troubleshooting immediate issues, visit [[Portal:Cloud_VPS/Admin/Troubleshooting]]. | ||
[[File: | [[File:OpenStack_at_WMCS.pdf|thumb|Slides from a brief presentation about WMCS OpenStack architecture]] | ||
== Cloud VPS Eqiad (Ashburn, VA) == | == Cloud VPS Eqiad (Ashburn, VA) == | ||
=== Horizon === | |||
Most users will manage their virtual servers using [[Horizon]]. Horizon is an upstream OpenStack web interface for the OpenStack APIs. Our Horizon site also includes several custom dashboards to access special WMCS features not available in stock Horizon. | |||
Horizon is hosted on labweb1001.wikimedia.org and labweb1002.wikimedia.org and can be accessed at https://horizon.wikimedia.org. | |||
Individual user accounts on WMCS can also be created via Striker which is at https://toolsadmin.wikimedia.org. Currently any account created there is automatically added to the Tools project. | |||
=== Controller === | === Controller === | ||
The | The OpenStack controller box (currently named 'labcontrol1001') runs the Glance and Keystone services, as well as nova-conductor and nova-scheduler. It is also the preferred place to access the OpenStack command-line client. | ||
A second server, labcontrol1002, serves as a hot spare for labcontrol1001. | A second server, labcontrol1002, serves as a hot spare for labcontrol1001. | ||
=== Network === | === Network === | ||
The network node ('labnet1001') hosts the nova-network service. We currently run a single | The network node ('labnet1001') hosts the nova-network service. We currently run a single cloud-wide network that supports all lab nodes and projects. In the near future we will move to [http://docs.openstack.org/developer/neutron/ Openstack Neutron] for our network setup. | ||
Labnet1001 also hosts the nova-api service. | Labnet1001 also hosts the nova-api service. | ||
Labnet1002 serves as a hot-spare for labnet1001. Switching network service between the two hosts causes cloud-wide network downtime and requires several [[Portal:Cloud_VPS/Admin/Troubleshooting#Fail-over | delicate steps]]. | |||
=== Virtualization === | === Virtualization === | ||
There are currently 22 virtualization nodes, named labvirt1001-1022, all running in eqiad: | |||
* 1001-1009 are high-powered multi-CPU [[HP_DL380p | HP servers]]; each of them hosts dozens of virtual machines. | |||
* 1010-1022 are similar to 1001-1009 but with large SSD raids. | |||
* 1019 and 1020 are used to host VMs that are themselves part of the Cloud infrastructure (e.g. database servers). | |||
* Some labvirts (as of 2015-05-10, labvirt1018, 1021, and 1022) are always reserved as emergency spares; we try to keep around 10% excess capacity at all times. | |||
Labvirt hosts can be pooled or depooled in Puppet Hiera using the profile::openstack::main::nova::scheduler_pool setting. | |||
=== Storage === | === Storage === | ||
Some VPS projects use shared NFS storage; most do not. Options for each project are are: | |||
* Each member of a project has a project-wide shared home directory. | * Each member of a project has a project-wide shared home directory. | ||
* | * The project has a public shared volume, generally mounted to /data/project | ||
All of the above are hosted on | All of the above are hosted on various NFS servers, labstore1xxx. | ||
=== monitoring === | === monitoring === | ||
Most OpenStack services and related things are monitored in icinga just like other production services. | |||
VMs in the 'Tools' and 'Deployment-Prep' projects are monitored with [http://shinken.wmflabs.org Shinken]. | |||
=== ldap === | === ldap === | ||
Ldap is used for services throughout the WMF. | Ldap is used for services throughout the WMF; the same Ldap database keeps track of project management and ssh keys for logins on VPS servers. Ldap is hosted on seaborgium and neptunium; The LDAP server software is openldap. | ||
Each CPS instance has an /etc/ldap.conf file (managed by puppet) that maintains info about the ldap servers. | |||
=== dns === | === dns === | ||
| Line 68: | Line 66: | ||
[[Portal:Cloud_VPS/DNS|DNS]] is handled by PowerDNS. Private DNS entries (e.g. foo.eqiad.wmflabs) are created via Designate Sink and stored in a PDNS server using a mysql backend. Public DNS entries are created via Horizon and the designate API. | [[Portal:Cloud_VPS/DNS|DNS]] is handled by PowerDNS. Private DNS entries (e.g. foo.eqiad.wmflabs) are created via Designate Sink and stored in a PDNS server using a mysql backend. Public DNS entries are created via Horizon and the designate API. | ||
[[File: | [[File:Wmcs dns.pdf|thumb|DNS in WMCS]] | ||
Revision as of 13:56, 8 September 2018
Cloud VPS is a virtualization cloud that uses OpenStack Compute. Base images are managed with Glance and authentication uses LDAP-backed Keystone.
Cloud VPS currently runs in a single datacenter in Ashburn, Virginia. In the future it will span two or more datacenters, with a slightly different configuration in each.
For troubleshooting immediate issues, visit Portal:Cloud_VPS/Admin/Troubleshooting.
Cloud VPS Eqiad (Ashburn, VA)
Horizon
Most users will manage their virtual servers using Horizon. Horizon is an upstream OpenStack web interface for the OpenStack APIs. Our Horizon site also includes several custom dashboards to access special WMCS features not available in stock Horizon.
Horizon is hosted on labweb1001.wikimedia.org and labweb1002.wikimedia.org and can be accessed at https://horizon.wikimedia.org.
Individual user accounts on WMCS can also be created via Striker which is at https://toolsadmin.wikimedia.org. Currently any account created there is automatically added to the Tools project.
Controller
The OpenStack controller box (currently named 'labcontrol1001') runs the Glance and Keystone services, as well as nova-conductor and nova-scheduler. It is also the preferred place to access the OpenStack command-line client.
A second server, labcontrol1002, serves as a hot spare for labcontrol1001.
Network
The network node ('labnet1001') hosts the nova-network service. We currently run a single cloud-wide network that supports all lab nodes and projects. In the near future we will move to Openstack Neutron for our network setup.
Labnet1001 also hosts the nova-api service.
Labnet1002 serves as a hot-spare for labnet1001. Switching network service between the two hosts causes cloud-wide network downtime and requires several delicate steps.
Virtualization
There are currently 22 virtualization nodes, named labvirt1001-1022, all running in eqiad:
- 1001-1009 are high-powered multi-CPU HP servers; each of them hosts dozens of virtual machines.
- 1010-1022 are similar to 1001-1009 but with large SSD raids.
- 1019 and 1020 are used to host VMs that are themselves part of the Cloud infrastructure (e.g. database servers).
- Some labvirts (as of 2015-05-10, labvirt1018, 1021, and 1022) are always reserved as emergency spares; we try to keep around 10% excess capacity at all times.
Labvirt hosts can be pooled or depooled in Puppet Hiera using the profile::openstack::main::nova::scheduler_pool setting.
Storage
Some VPS projects use shared NFS storage; most do not. Options for each project are are:
- Each member of a project has a project-wide shared home directory.
- The project has a public shared volume, generally mounted to /data/project
All of the above are hosted on various NFS servers, labstore1xxx.
monitoring
Most OpenStack services and related things are monitored in icinga just like other production services.
VMs in the 'Tools' and 'Deployment-Prep' projects are monitored with Shinken.
ldap
Ldap is used for services throughout the WMF; the same Ldap database keeps track of project management and ssh keys for logins on VPS servers. Ldap is hosted on seaborgium and neptunium; The LDAP server software is openldap.
Each CPS instance has an /etc/ldap.conf file (managed by puppet) that maintains info about the ldap servers.
dns
DNS is handled by PowerDNS. Private DNS entries (e.g. foo.eqiad.wmflabs) are created via Designate Sink and stored in a PDNS server using a mysql backend. Public DNS entries are created via Horizon and the designate API.