You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org
This workshop outlines how to use Kubernetes (minikube) and Docker within the Wikimedia Foundation (WMF). Kubernetes (k8s, a numeronym similar to I18n or l8n), is an open-source orchestration system. In practice, Kubernetes is used to automate a computer application deployment, support application scaling, and provide facilities for application management.
The workshop grants you:
- A hands-on understanding of WMF's primary use-cases of Kubernetes.
- An understanding of how WMF employs the tool in the production environment.
- An in-depth review of how to use WMF’s Kubernetes to run your services.
- A practical experience in running sample applications on k8s.
This document is aimed at:
- Engineers who are new to Kubernetes and want a basic hands-on introduction to how to use WMF's Kubernetes to run their services.
What this is not
This workshop is not a complete tutorial on practicing with Kubernetes.
What this is
This is a living document tailored to outline the specific ways WMF uses Kubernetes.
Kubernetes (k8s) is an open-source container orchestration system. An orchestration system automates deployment and management of application. Here is an example of what Kubernetes can do for you:
- Deploy your application on any infrastructure (on-premise, hybrid, public cloud...)
- Autoscale it based on demand
- Perform zero-downtime updates and rollbacks
- Restart, replace and reschedule failing containers
- Secret and configuration management
- And much more!
In order for an app to be deployable on Kubernetes, it needs to be packaged as a container. However, Kubernetes can't work with containers directly, but wraps them in an additional layer of functionality, creating a Pod. A Pod is the minimal deployable unit that you can create and manage in Kubernetes. In this workshop, you will get some hands-on experience working with Kubernetes in general, as well as gain an understanding of how it is used in WMF.
When planning for large-scale application deployments, k8s is a viable option. However, k8s is not the best solution for all environments, see the Additional Resources section for more information.
The Wikimedia Foundation uses k8s in several scenarios. The Wikimedia Cloud Services (WMCS) team runs k8s in Toolforge, while the SRE ServiceOps team runs k8s as a microservice platform. Prerequisites
To navigate through this workshop, WMF recommends that you possess the following:
- A basic programming and Linux knowledge.
- Basic knowledge of containerization and Docker.
- Some familiarity with automation tools such as Ansible.
- Basic knowledge of GitHub and Gerrit.
Using a Linux machine is best for this workshop. The workshop also provides troubleshooting guides for Mac and Windows users.
You can use a provisioned Virtual Machine (VM) from any cloud provider or WMCS. An ideal VM should possess the following:
- 2 CPUs
- 4 GB Memory
- 40 GB disk
In this workshop, you will use a local installation of k8s, minikube, Docker, Python.
Docker is a containerization tool used to package applications and their dependencies into containers. This guide outlines steps on how to install Docker on your local machine.
- Linux (Ubuntu) Installation
- macOS Installation
- Windows Installation
- Allow the current user to run Docker without root permissions by adding your user to the docker group:
$ sudo usermod -aG docker $USER && newgrp docker
To test your Docker installation, run the hello-world application:
$ docker run hello-world Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 2db29710123e: Pull complete Digest: sha256:10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67 Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly.
Expect a similar output as the code block above if your installation was successful.
Note: Ensure the Docker daemon is running if you are using a macOS machine. Start the Docker daemon by clicking on your Docker application before running the hello-world application.
Verify your installation:
$ ansible all -m ping --ask-pass SSH password: [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
At the prompt for the SSH password, press the Enter key.
Note: macOS you can also install Ansible by running:
$ brew install ansible
This guide outlines steps to install kubectl, a Kubernetes CLI tool used to run commands against clusters. To quickly find common kubectl commands, use this cheat sheet. From the links given below, install Kubernetes on your local machine:
minikube is used to run a Kubernetes cluster locally (on personal computers). Highlighted below are ways to install this tool on various operating systems.
- Ensure the Docker daemon is running.
- Enable Kubernetes on Docker Desktop: this can be done by:
- Navigating to the settings page.
- Clicking on Kubernetes.
- Ticking the box for the option Enable Kubernetes
Verify your installation:
$ minikube start 😄 minikube v1.25.2 on Darwin 12.3.1 ✨ Automatically selected the docker driver 👍 Starting control plane node minikube in cluster minikube 🚜 Pulling base image ... 💾 Downloading Kubernetes v1.23.3 preload ... > preloaded-images-k8s-v17-v1...: 505.68 MiB / 505.68 MiB 100.00% 2.20 MiB > gcr.io/k8s-minikube/kicbase: 379.06 MiB / 379.06 MiB 100.00% 1.28 MiB p/ 🔥 …………………………………………………….. namespace by default $ kubectl cluster-info Kubernetes control plane is running at https://127.0.0.1:50198 CoreDNS is running at https://127.0.0.1:50198/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Getting a URL response implies that kubectl is correctly configured to access your cluster.
Verify your kubectl installation:
$ kubectl get nodes NAME STATUS ROLES AGE VERSION minikube Ready control-plane,master 64s v1.23.3 $ kubectl run helloworld --image=hello-world pod/helloworld created $ kubectl logs helloworld Hello from Docker! This message shows that your installation appears to be working correctly. ………………………………………………………………………… For more examples and ideas, visit: https://docs.docker.com/get-started/ $ kubectl get pods NAME READY STATUS RESTARTS AGE helloworld 0/1 CrashLoopBackOff 2 (23s ago) 42s $ minikube stop ✋ Stopping node "minikube" ... 🛑 Powering off "minikube" via SSH ... 🛑 1 node stopped.
This workshop contains eleven modules to acquaint you with Kubernetes and its use in WMF’s production environments. The workshop introduces the following concepts:
- Executing a generic k8s cluster on Wikimedia Cloud Services (WMCS).
- Using WMF's deployment pipeline tools, namely Gerrit, Blubber, and Continuous Integration/Continuous Deployment (CI/CD) systems.
The appendix contains links to Docker tutorials that provide the necessary foundational knowledge. Each module should take around two or three hours to complete.
The workshop presents you with a hands-on coding experience, plenty of examples, and references to unblock you when you experience blockers.
Verify the following before attempting the workshop:
- Ensure you have Python running on your machines. Linux OS users can install Python 3 by running the following commands:
$ sudo apt update $ sudo apt install software-properties-common $ sudo add-apt-repository ppa:deadsnakes/ppa $ sudo apt update $ sudo apt install python3 $ python3 --version Python3.7.3
Your Python version output might be different from the above output. However, the output’s format remains the same.
- Most virtual machines come preinstalled with Git. However, you can install Git on your VM or configure your laptop’s terminal to run git commands. To verify whether you have GitHub installed on your VM, run the following command:
$ git version -bash: Git: command not found
If you get the above output, you have to install Git; otherwise, you have Git installed on your VM. You can install Git by running the following commands:
$ sudo apt update $ sudo apt install git-all
To verify your installation, run:
$ git version git version 2.20.1
- Lastly, install pip on your VM:
$ sudo apt update $ sudo apt install python3-pip $ pip3 --version pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)
- Run the hello-minikube application
Mac users on Big Sur should use this for the base setup.
A previous version of this workshop can be found here.
This video by Cruise, presented at Kubecon, talks about the number of clusters and nodes. At the same time, this video highlights the achieved benefits. k8s is not the best solution for all environments, and this article gives some interesting points.