You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Difference between revisions of "Help:Cloud Services Introduction"

From Wikitech-static
Jump to navigation Jump to search
imported>Srodlund
imported>JJMC89
m (Reverted edits by Josh14 (talk) to last revision by EpicPupper)
(34 intermediate revisions by 18 users not shown)
Line 1: Line 1:
{{notice | wmflabs.org and wmcloud.org redirect here. You might be looking for the [[:Category:Toolforge tools|Lists of Toolforge tools]] or the [[toolforge:openstack-browser/project/|List of Cloud VPS projects]].}}
[[File:What is Cloud Services? poster.pdf|thumb|upright=1.3|Poster-format overview]]
[[File:What is Cloud Services? poster.pdf|thumb|upright=1.3|Poster-format overview]]


Line 5: Line 7:
===What is Wikimedia Cloud Services (WMCS)?===
===What is Wikimedia Cloud Services (WMCS)?===


'''Wikimedia Cloud Services''' ('''WMCS''') empowers technical contributions to the Wikimedia software world. WMCS is a flexible computing ecosystem built on [[W:OpenStack|OpenStack]], [[W:Oracle Grid Engine|GridEngine]], and [[W:Kubernetes|Kubernetes]].  
'''Wikimedia Cloud Services''' ('''WMCS''') provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. WMCS is a computing ecosystem built on [[W:OpenStack|OpenStack]], [[W:Oracle Grid Engine|GridEngine]], and [[W:Kubernetes|Kubernetes]].  
 
WMCS products and services are available for use by anyone connected with the [[meta:Wikimedia movement|Wikimedia movement]] without charge. Support and administration of the WMCS resources is provided by a [[Mw:Wikimedia Cloud Services team|Wikimedia Foundation Cloud Services team]] and [[wmf:Volunteer_opportunities|Wikimedia movement volunteers]]. We maintain a [[Help:Glossary|Glossary]] of related terminology.
 
🎬 '''Video''': [https://media.ccc.de/v/36c3-77-wikimedia-cloud-services-introduction  Wikimedia Cloud Services introduction]


WMCS products and resources are available for use by anyone connected with the [[meta:Wikimedia movement|Wikimedia movement]]. Support and administration of the WMCS resources is provided by a [[Mw:Wikimedia Cloud Services team|Wikimedia Foundation Cloud Services team]] and [https://wikimediafoundation.org/wiki/Volunteer_opportunities Wikimedia movement volunteers].
📣 '''Slides''': [[commons:File:Introduction_to_Wikimedia_Cloud_Services_-_Wikimania_Hackathon_2019_Stockholm_Sweden.pdf|An introduction to Cloud Services presentation from Wikimania Stockholm 2019]]


=== WMCS history ===
=== WMCS history ===


From 2011 until early 2017, WMCS was known as "Wikimedia Labs." The term 'Labs' was used to refer to a number of different [[Labs labs labs|components]], and clarification was required. In 2017, the project was reorganized. The former Wikimedia Foundation Labs team and the Tool Labs Support team joined together to create the Wikimedia Cloud Services team.  
From 2011 until early 2017, WMCS was known as "Wikimedia Labs." The term 'Labs' was used to refer to a number of different [[Labs labs labs|components]], and clarification was required. In 2017, the project was reorganized. The former Wikimedia Foundation Labs team and the Tool Labs Support team joined together to create the Wikimedia Cloud Services team.  
=== Renaming of products ===
Efforts are underway to improve the [[phab:phame/post/view/59/labs_and_tool_labs_being_renamed/| language and branding]] surrounding the products and services we offer.
Working titles are now in use across some documentation. The generic compute/'Labs' environment for running virtual instances is now referred to as "Cloud VPS," and the Tool Labs PaaS environment is now referred to as "Toolforge."


{{anchor|WMCS Products}}
{{anchor|WMCS Products}}


== WMCS products ==
== WMCS products and services ==
 
* [[Portal:Cloud VPS|Cloud VPS]] provides collaboratively owned collections of virtual private servers where users develop and maintain software projects that help the Wikimedia movement.
 
* [[Portal:Toolforge|Toolforge]] provides a shared hosting/platform as a service environment for running bots, webservices, scheduled jobs, and data analysis.
 
* [[Portal:Data Services|Data Services]] is a collection of products including private-information-redacted copies of Wikimedia's production wiki databases and access to [[Dumps.wikimedia.org|Wikimedia Dumps]]


{{anchor|What product should I use?}}
{{anchor|What product should I use?}}
{| class="wikitable sortable"
|+ WMCS Products
! Service
! Product
! Description
! Use
! Support Level
|-
| [[:en:Virtual_private_server|VPS]]
| [[Portal:Cloud VPS|Cloud VPS]]
| Provides collaboratively owned collections of virtual private servers where users develop and maintain software projects that help the Wikimedia movement.
| Use this to run full virtual instances.
| You are willing to administer instances on your own. We can provide quota to do so.  
|-
| [[:en:Platform as a service|PaaS]]
| [[Portal:Toolforge|Toolforge]]
| Provides a shared hosting/platform-as-a-service environment for running bots, webservices, scheduled jobs, and data analysis.
| Run a specific webservice, scheduled job, or perform analysis.
| You do not want to or are not able to manage a full virtual environment.
|-
| [[:en:Data_as_a_service|DaaS]]
| [[Portal:Data Services|Data Services]]
| A collection of products including private-information-redacted copies of Wikimedia's production wiki databases and access to [[Dumps.wikimedia.org|Wikimedia Dumps]].
| Create replicas of the production databases and other data for analysis and experimentation.  
| The [[quarry:|Quarry service]] provides database access via a web interface. Some DaaS resources may need to be requested for specific VPS projects.
|}


== Which product should I use? ==
=== Renaming of products and services ===
We are in the process of changing the [[phab:phame/post/view/59/labs_and_tool_labs_being_renamed/| language and branding]] of the products and services we offer. You may find some outdated titles and names in WMCS documentation. Edits are welcome!


=== VPS → [[Portal:Cloud VPS|Cloud VPS]] ===
== Participating with WMCS ==
If you need to run full virtual instances and are willing to administer them on your own we can provide quota to do so. 


=== PaaS → [[Portal:Toolforge|Toolforge]] ===
=== Sign up for services ===
If you need to run a specific webservice, scheduled job, or perform analysis but do not want to or are not able to manage a full virtual environment, this is the best place.


=== DaaS → [[Portal:Data Services|Data Services]] ===
To access and contribute to Cloud Services projects and tools, you will need the following accounts:
We provide replicas of the production databases and other data for analysis and experimentation.  DaaS resources are available from both the VPS and the PaaS environments, though some of them may need to be requested for a specific VPS Project. In addition, the [https://quarry.wmflabs.org/ Quarry service] provides database access via a web interface without project setup.


== Terms of use ==
*'''Wikimedia account''' - this account is the single user login or SUL account you use to contribute to Wikipedia and its sister projects.
* All WMCS resources are subject to the [[Wikitech:Labs_Terms_of_use|Wikimedia Labs Terms of Use]] and the [[mw:Code of Conduct|Code of Conduct for technical spaces]].
*[[Help:Create_a_Wikimedia_developer_account|'''Wikimedia developer account''']] - this account is used log into this wiki, Toolforge, Cloud VPS, Gerrit and other protected Wikimedia Services.
* Toolforge tools must be [[w:Open-source software|open source software]] licensed under an [http://opensource.org/licenses OSI approved license].
* '''[[mw:Gerrit|Gerrit]]'''- our code review system; where our repositories (repos) live. Note that while [[mw:Gerrit/GitHub/GitHub]] contains many of our public repos, you can only make pull requests for Cloud Services projects via Gerrit. Other Wiki projects may use GitHub exclusively.
* Toolforge and Cloud VPS projects must not collect, store, or share private data or personally identifiable information, such as user names, passwords, or IP addresses, except when complying with the conditions listed in the [[Wikitech:Labs_Terms_of_use|Wikimedia Labs Terms of Use]].
* '''[[mw:Phabricator|Phabricator]]'''- our project management system; for opening tickets, suggesting features, and talking about our plans for the next quarter.
* '''[[mw:MediaWiki on IRC|IRC]]''' - live chat channels. We have several channels related to our cloud servers, but the main channel is #wikimedia-cloud. Deployment also frequently uses #wikimedia-serviceops.
* The ability to [[Help:Access|access instances]] in the WMCS environment.


== Joining and participating ==
===Review the terms and conditions===


[[Help:Terminology|Wikitech Account Holders]] are governed by the [[Project:Labs Terms of use|Labs terms of use]], the [[Help:Rules|Rules]], and the [[mw:Code of Conduct|Code of Conduct for technical spaces]]. The [[mw:Wikimedia Labs/Agreement to disclosure of personally identifiable information|agreement to disclosure of personally identifiable information]] covers [[Help:Terminology|End-Users]].
Second, make sure to review and agree to our terms and conditions. [[Help:Terminology|Account holders]] who plan to use WMCS resources and products must read and agree to the following:


Read our [[Help:Getting_Started|Getting Started]] guide for detailed information on creating accounts and becoming active in our projects.
* [[Wikitech:Cloud Services Terms of use|Wikimedia Cloud Services Terms of Use]]  
* [[mw:Code of Conduct|Code of Conduct for technical spaces]]
* [[mw:Wikimedia Labs/Agreement to disclosure of personally identifiable information|Agreement to disclosure of personally identifiable information]] (covers [[Help:Terminology|End-Users]]).


== Support and administration ==
Please pay close attention to the following terms for '''Toolforge and Cloud VPS:'''


We use Phabricator ([[phab:project/profile/832/|workboard]]), an [[Help:IRC|IRC]] channel ({{Irc|wikimedia-cloud}}), and email list ([https://lists.wikimedia.org/mailman/listinfo/cloud cloud]) as primary support channels.
* Toolforge tools must be [[w:Open-source software|open source software]] licensed under an [http://opensource.org/licenses OSI approved license].
* Toolforge and Cloud VPS projects must not collect, store, or share private data or personally identifiable information, such as user names, passwords, or IP addresses, except when complying with the conditions listed in the [[Wikitech:Cloud Services Terms of use|Wikimedia Cloud Services Terms of Use]].


Wikimedia Cloud Services are a joint effort between Wikimedia Foundation staff and many dedicated volunteers:
{{:Help:Cloud Services communication}}
* [[mw:Wikimedia Cloud Services team|Wikimedia Cloud Services team]]
* [[mw:Wikimedia Technical Operations|Wikimedia Technical Operations]]
* [[meta:Community Liaisons|Community Liaisons]]
* [[Help:Toolforge/Toolforge standards committee|Toolforge standards committee]]
* [[phab:project/members/832/|Phabricator participants]]
* [[toollabs:openstack-browser/project/|Cloud VPS project]] maintainers
* [https://tools.wmflabs.org/?list Toolforge tool] maintainers
* and many more people who help by creating and maintaining documentation on this wiki and others in the Wikimedia movement
 
{{anchor|Discovery}}


== Learn more ==
== Learn more ==


[[File:WMCS components.svg|thumb|upright=1.3|WMCS components]]
* [[Help:FAQ | Cloud VPS and Toolforge At-A-Glance]]: This page provides a basic introduction to '''Cloud VPS''' and '''Toolforge'''.
 
* [[Portal:Cloud_VPS | Cloud VPS Portal]]: Information about Cloud VPS and links to help and technical documentation.
* Read our glossary of [[Help:Terminology|terminology]]
* [[Portal:Toolforge | Toolforge Portal]]: Information about Toolforge and links to help and technical documentation.
* Browse existing VPS projects using [[toollabs:openstack-browser/|OpenStack Browser]].
* [[Portal:Data_Services | Data Services Portal]]: Information about Data Services and links to help and technical documentation.
* Project administration is done through the OpenStack dashboard known as [https://horizon.wikimedia.org/ Horizon].
* Toolforge (Tools) has a [[toollabs:|homepage]] and a [[toollabs:?list|directory of Tools]]
 
;Other explanations
* [[wmfblog:/2017/09/11/introducing-wikimedia-cloud-services/|Introducing the Cloud Services Team: What we do, and how we can help you]] blog entry (September 2017)
* [[:Media:WMCS_Presentation.pdf|What is Wikimedia Cloud Services?]] presentation slides (12 pages)
* [[c:File:Introduction to Wikimedia Cloud Services.webm|Video presentation: Introduction to Wikimedia Cloud Services, at Wikimania 2017]] (56 minutes)


[[Category:Documentation]]
[[Category:Cloud Services]]

Revision as of 20:44, 13 August 2021

Poster-format overview

WMCS project overview

What is Wikimedia Cloud Services (WMCS)?

Wikimedia Cloud Services (WMCS) provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. WMCS is a computing ecosystem built on OpenStack, GridEngine, and Kubernetes.

WMCS products and services are available for use by anyone connected with the Wikimedia movement without charge. Support and administration of the WMCS resources is provided by a Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. We maintain a Glossary of related terminology.

🎬 Video: Wikimedia Cloud Services introduction

📣 Slides: An introduction to Cloud Services presentation from Wikimania Stockholm 2019

WMCS history

From 2011 until early 2017, WMCS was known as "Wikimedia Labs." The term 'Labs' was used to refer to a number of different components, and clarification was required. In 2017, the project was reorganized. The former Wikimedia Foundation Labs team and the Tool Labs Support team joined together to create the Wikimedia Cloud Services team.

WMCS products and services

WMCS Products
Service Product Description Use Support Level
VPS Cloud VPS Provides collaboratively owned collections of virtual private servers where users develop and maintain software projects that help the Wikimedia movement. Use this to run full virtual instances. You are willing to administer instances on your own. We can provide quota to do so.  
PaaS Toolforge Provides a shared hosting/platform-as-a-service environment for running bots, webservices, scheduled jobs, and data analysis. Run a specific webservice, scheduled job, or perform analysis. You do not want to or are not able to manage a full virtual environment.
DaaS Data Services A collection of products including private-information-redacted copies of Wikimedia's production wiki databases and access to Wikimedia Dumps. Create replicas of the production databases and other data for analysis and experimentation.   The Quarry service provides database access via a web interface. Some DaaS resources may need to be requested for specific VPS projects.

Renaming of products and services

We are in the process of changing the language and branding of the products and services we offer. You may find some outdated titles and names in WMCS documentation. Edits are welcome!

Participating with WMCS

Sign up for services

To access and contribute to Cloud Services projects and tools, you will need the following accounts:

  • Wikimedia account - this account is the single user login or SUL account you use to contribute to Wikipedia and its sister projects.
  • Wikimedia developer account - this account is used log into this wiki, Toolforge, Cloud VPS, Gerrit and other protected Wikimedia Services.
  • Gerrit- our code review system; where our repositories (repos) live. Note that while mw:Gerrit/GitHub/GitHub contains many of our public repos, you can only make pull requests for Cloud Services projects via Gerrit. Other Wiki projects may use GitHub exclusively.
  • Phabricator- our project management system; for opening tickets, suggesting features, and talking about our plans for the next quarter.
  • IRC - live chat channels. We have several channels related to our cloud servers, but the main channel is #wikimedia-cloud. Deployment also frequently uses #wikimedia-serviceops.
  • The ability to access instances in the WMCS environment.

Review the terms and conditions

Second, make sure to review and agree to our terms and conditions. Account holders who plan to use WMCS resources and products must read and agree to the following:

Please pay close attention to the following terms for Toolforge and Cloud VPS:

Communication and support

We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.

Communicate with us
Connect Best for
Phabricator Workboard #Cloud-Services Task tracking and bug reporting
IRC Channel #wikimedia-cloud connect
Telegram bridge
mattermost bridge
General discussion and support
Mailing List cloud@ Information about ongoing initiatives, general discussion and support
Announcement emails cloud-announce@ Information about critical changes (all messages mirrored to cloud@)
News wiki page News Information about major near-term plans
Cloud Services Blog Clouds & Unicorns Learning more details about some of our work
Wikimedia Technical Blog techblog.wikimedia.org News and stories from the Wikimedia technical movement

Learn more