You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Help:Cloud VPS Instances

From Wikitech-static
(Redirected from Instances)
Jump to navigation Jump to search

This page contains information on how to set up a Cloud VPS instance.

What is a Cloud VPS instance?

An instance is a virtual machine (VM). Every instance belongs to a Cloud VPS project.

Difference between a Cloud VPS project and an instance

  • A Cloud VPS project can be defined as the group of users and resources (like instances, security groups, floating IPs, Puppet groups, etc.) associated with a piece of software hosted on Cloud VPS.
  • An instance is just one resource that a Cloud VPS project might use.

Why create a Cloud VPS instance?

Cloud VPS is a flexible option for those who need it. It is equivalent to having your own server(s).

Before requesting a Cloud VPS instance, we recommend you consider using Toolforge instead. Toolforge has many of the same features as Cloud VPS, and Toolforge users do not have to manage the full virtual environment. Toolforge users also have access to toolsdb, wiki replicas, and a number of other services.

See Help:Cloud Services Introduction to help decide which service is right for you.

Who can create a Cloud VPS instance?

Project admins (sometimes written projectadmins). You must be a Project admin in a project to create and manage its instances.

What are the responsibilities of project admins?

Project admins must subscribe to the cloud-announce and cloud mailing lists and follow the maintenance or communication actions requested by the WMCS Admins. Failure to respond may result in unexpected loss of data or service interruptions.

Project admins should delete unused instances and notify other admins when a project is no longer active. See Cloud VPS instance lifecycle for more details.

Requesting Project admin rights

To create instances, individuals must have project admin rights on the specific project they want to have instances created for. The following people can grant Project admin rights:

  • Project admins. Project admins can grant project admin rights on projects they belong to. You can find Project admins listed on the individual pages for their projects.
  • Cloud VPS admins. Cloud admins, including Wikimedia Cloud Services staff and trusted community volunteers, can manage membership and Project admin status in any project.

Contacting admins
Project admins and members of the Cloud admin group can be contacted through the 'email a user' function or through their talk page on Wikitech. Please note: Many members of the Wikitech community are volunteers who may not check their email or talk pages regularly. You may need to check with more than one admin before you receive a response.

Project access rights
Learn more about Cloud services user roles and rights.

Before you create a Cloud VPS instance

  • Be a Project admin in the project you intend to create an instance for.
  • Make sure your project has an appropriate security group for the instance's networking needs.
  • Be aware of your project's quotas (usage limits).
  • Understand the Cloud VPS instance lifecycle.

Create a Cloud VPS instance

Pre-requisites: Setup accounts and enable multi-factor authentication

  1. Head over to this wiki page to create a Developer Account.
  2. Associate your account with a project. You can follow the steps listed on this [page]
  3. In your developer account enable Multi-Factor Authentication by taking the following steps:
  • First, click on Preferences.
  • Then click on Manage, listed under the Two Factor-authentication option.
File:2FA.png
Two Factor-authentication option
  • Select Enable TOTP (one-time token) method.
  • You can make use of the Google Authenticator App or the Authy app (which has some advantages like encrypted backups). If you download the Google Authenticator app on your phone and click on the “+” sign.
  • Select the option ‘Scan QR code’.
  • Fill out the unique code on the website.

Working with Cloud VPS Instances

  • Log in to https://horizon.wikimedia.org/.
  • Switch to the Project where you will host your VM. This option is available at the top-left of your Cloud VPC portal.
File:Instances.png
Instances section
  • The interface should change and you should be able to view, launch and delete Instances. This implies that you have admin privileges.The highlighted column at the top left of the image above, provides you an option to select a project scope you can provision your machine in. You can be in one or multiple projects within openstack.
File:Instances 2.png
Instances page when you have admin privileges

The steps below will guide you through setting up a VM on this interface:

  • Fill out the form as presented below. Ignore the Availability zone option and leave Count as 1. Then click Next.
File:Launch Instance - Step 1.png
Launch Instance - Step 1
  • Select an Operating System Image from the Available section.
File:Launch Instance - Step 2.png
Launch Instance - Step 2
  • Do this by clicking the up-arrow icon to move the image into the Allocated section, and click Next.
File:Launch Instance - Step 2b.png
Launch Instance - Step 2b
  • Select the Flavor. The flavor influences how much RAM, CPU and disk your VM gets. You are limited to a few predetermined combinations here. Click Next.
File:Launch Instance - Step 3.png
Launch Instance - Step 3
  • Leave the Security Groups section as default. Click Next.
  • Leave Server Groups as default.
File:Launch Instance - Step 5.png
Launch Instance - Step 5
  • Clicking the Launch Instance button launches your new instance in the Spawning state, as seen in the instance overview.
File:Launch Instance - Step 6.png
Launch Instance - Step 6
  • Click on an instance’s name to view its details. The output should be similar to the image below:
File:Launch Instance - Step 7.png
Launch Instance - Step 7

To configure this instance with a puppet role for a specific job, click Puppet Configuration. You can apply puppet classes and, optionally, Hiera config values:

File:Launch Instance - Step 7b.png
Launch Instance - Step 7b

Puppet Configuration for Cloud VPS instances

  • Cloud VPS uses Puppet to manage instance configurations.
  • If you are a sudo user for the project, you can run sudo puppet agent -tv or wait until puppet runs on your instance.

Logging into VMs with SSH

  • Now to set up an SSH key on your service, you have to head back to your [Developer Account] and click on OpenStack under Preferences. [[1]] wiki page outlines the steps to create a public SSH key.
File:Launch Instance - SSH.png
Launch Instance - SSH
  • SSH into your instance created on your Cloud VPC account. This wiki sheds more light on the various methods by which you can log into your instance.

$ ssh -J <your-shell-name>@bastion.wmcloud.org <your-shell-name>@<your-instance>.<your-project>.eqiad1.wikimedia.cloud

where,

your-shell-name is the name you picked when signing up on Wikitech.

your-instance is the name you picked when creating the VM.

your-project is the name of the project you are building under

Note:

  • Copy and paste only the alpha-numeric texts generated by your SSH agent in the text box provided on your developer account.


Instance information

Information type Action
Instance Details The name, zone, and count of your instance The name will also be your hostname. Name your instance using the following format project-instance, e.g, the project name is myproject; the instance is coolbot; the instance name is myproject-coolbot
Instance Source The version of the operating system (Debian Stretch, Debian Buster, etc). Also known as a base image.
Instance Flavor Contains a range of possible instance types indicating RAM, number of CPUs, and Hard Drive space. Your most likely choice will be at the top of the list of system images available. See current flavors in following chart. This decision is not permanent, you can allocate more resources later by using the red "resize instance" button.
Security Groups Contains a list of security groups At minimum, the default security group should be selected.*
Server Groups Contains a list of server groups Server groups allow you to define logical groups of instances with advanced scheduling features. Note: avoid using the affinity server group policy. See the linked help page for why.

* If the Project admin for the project has not created any additional security groups, besides the default group, you will only see the default security group listed.

Instance Sizes and Flavors

Cloud VPS sizes*
Name VCPUS RAM ROOT DISK EPHEMERAL DISK PUBLIC
g3.cores1.ram2.disk20 1 2 GB 20 GB 0 GB Yes
g3.cores2.ram4.disk20 2 4 GB 20 GB 0 GB Yes
g3.cores4.ram8.disk20 4 8 GB 20 GB 0 GB Yes
g3.cores8.ram16.disk20 8 16 GB 20 GB 0 GB Yes
g3.cores16.ram16.disk20 16 16 GB 20 GB 0 GB Yes
g2.cores8.ram36.disk20 8 36 GB 20 GB 0 GB Yes

* Current as of June 2022

Setting up a test MediaWiki Server

See MediaWiki-Vagrant in Cloud VPS for more information.

Increase quotas for projects

Quotas are the usage limits for projects. Quotas refer to one or more of CPU, RAM, disk storage, number of VM instances and/or floating IPs. The default quota for a new project allows up to 8 instances.

Projects can have their quota increased following the instructions on the "Cloud-VPS (Quota-requests)" Phabricator task.

The Cloud VPS Instance lifecycle

Instances are allocated resources (storage, RAM, CPU, power). Over time, these resources need to be used or reclaimed. WMCS admins periodically check projects and instances to ensure they are being used by active projects. Instances will be removed for projects that have been determined inactive.

Process for removing Cloud VPS instances

  1. Notice will be sent to cloud via cloud-announce at least 2 weeks before any expected action is taken. Emergencies may require more immediate intervention.
  2. If possible, instances will be suspended or shut down for at least 2 weeks (a total of a month) before further unsolicited actions are taken to allow the tenant to notice.
  3. Inactive projects (those without instances or any responsive users) will be removed (along with their quota)

Backups of Cloud VPS instances

No backups will be kept by Wikimedia Cloud Services.

Local backups

You can backup your server by copying the files you are interested to save on your local machine. Of course, this is not an automated process and you will need to store the data you are saving. Also, be mindful that backups may be very big and so you will be transferring a lot of data, which among other things will take a long time.

If you want to backup your root disk on your VPS server to a local directory named backup you can do the following:

rsync -aXvz --stats --info=progress2 --rsync-path="sudo /usr/bin/rsync" --exclude-from=./excludes.txt <server>:/ backup/`

The paths listed in the file excludes.txt will be ignored. This gist contains a common list of excluded paths that you usually do not need to backup since they are temporary files, cache, backups and other system files.

Communication and support

We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.

Communicate with us
Connect Best for
Phabricator Workboard #Cloud-Services Task tracking and bug reporting
IRC Channel #wikimedia-cloud connect General discussion and support
Mailing List cloud@ Information about ongoing initiatives, general discussion and support
Announcement emails cloud-announce@ Information about critical changes (all messages mirrored to cloud@)
News wiki page News Information about major near-term plans
Blog Clouds & Unicorns Learning more details about some of our work

See Also