Kubernetes/Kubernetes Workshop

Overview

This workshop outlines how to use Kubernetes (minikube) and Docker within the Wikimedia Foundation (WMF). Kubernetes (k8s, a numeronym similar to I18n or l8n), is an open-source orchestration system. In practice, Kubernetes is used to automate a computer application deployment, support application scaling, and provide facilities for application management.

The workshop grants you:

• A hands-on understanding of WMF's primary use-cases of Kubernetes.
• An understanding of how WMF employs the tool in the production environment.
• An in-depth review of how to use WMF’s Kubernetes to run your services.
• A practical experience in running sample applications on k8s.

Audience

This document is aimed at:

• Developers
• Students
• Engineers who are new to Kubernetes and want a basic hands-on introduction to how to use WMF's Kubernetes to run their services.

What this is not

This workshop is not a complete tutorial on practicing with Kubernetes.

What this is

This is a living document tailored to outline the specific ways WMF uses Kubernetes.

Getting Started

Kubernetes (k8s) runs on top of multiple Linux systems that form a k8s cluster. The cluster comprises the management systems (control plane), nodes, and dedicated systems upon which the Linux-based applications execute.

In this workshop, you will deploy applications as Docker images and run these images as containers inside pods on the nodes. The cgroup kernel feature controls the application resources.

Kubernetes

When planning for large-scale application deployments, k8s is a viable option. However, k8s is not the best solution for all environments, see the Additional Resources section for more information.

The Wikimedia Foundation uses k8s in several scenarios. The Wikimedia Cloud Services (WMCS) team runs k8s in Toolforge, while the SRE ServiceOps team runs k8s as a microservice platform.

Prerequisites

To navigate through this workshop, WMF recommends that you possess the following:

• A basic Linux knowledge.
• Basic knowledge of Docker.
• A basic understanding of Ansible.
• Basic knowledge of GitHub and Gerrit.

Using a Linux machine is best for this workshop. The workshop also provides troubleshooting guides for Mac and Windows users.

You can use a provisioned Virtual Machine (VM) from any cloud provider or WMCS. An ideal VM should possess the following:

• 2 CPUs
• 4 GB Memory
• 40 GB disk

Skills

To complete this workshop's exercises, you should have the following skills:

• Basic programming knowledge.
• Basic understanding of containerization.
• Basic familiarity with automation tools.

Setting Up

In this workshop, you will use a local installation of k8s, minikube, Docker, Python.

Installing Docker

Docker is a containerization tool used to package applications and their dependencies into containers. This guide outlines steps on how to install Docker on your local machine.

• Linux (Ubuntu) Installation
• MacOs Installation
• Windows Installation
• Take care to add your user to the docker group:

$ sudo usermod -aG docker $USER && newgrp docker

To test your Docker installation, run the hello-world application:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

Expect a similar output as the code block above if your installation was successful.

Note: Ensure the Docker daemon is running if you are using a MacOS machine. Start the Docker daemon by clicking on your Docker application before running the hello-world application.

Installing Ansible

Ansible is an enterprise automation platform. This guide outlines steps on how to install Ansible on your local machine.

• Linux (Ubuntu) Installation
• MacOs Installation
• Windows Installation

Verify your installation:

$ ansible all -m ping --ask-pass
SSH password:
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

At the prompt for the SSH password, press the Enter key.

Note: MacOs you can also install Ansible by running:

$ brew install ansible

This method uses the brew installer instead of pip.

Installing Kubernetes

This guide outlines steps to install kubectl, a Kubernetes CLI tool used to run commands against clusters. To quickly find common kubectl commands, use this cheat sheet. From the links given below, install Kubernetes on your local machine:

• Linux Installation
• MacOs Installation
• Windows Installation

Installing minikube

minikube is used to run a Kubernetes cluster locally (on personal computers). Highlighted below are ways to install this tool on various operating systems.

• Linux Installation
• MacOS Installation
• Windows Installation

Note:

• Ensure the Docker daemon is running.
• Enable Kubernetes on Docker Desktop: this can be done by:
  • Navigating to the settings page.
  • Clicking on Kubernetes.
  • Ticking the box for the option Enable Kubernetes

Verify your installation:

$ minikube start
😄  minikube v1.25.2 on Darwin 12.3.1
✨  Automatically selected the docker driver
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.23.3 preload ...
    > preloaded-images-k8s-v17-v1...: 505.68 MiB / 505.68 MiB  100.00% 2.20 MiB
    > gcr.io/k8s-minikube/kicbase: 379.06 MiB / 379.06 MiB  100.00% 1.28 MiB p/
🔥 ……………………………………………………..
namespace by default

$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:50198
CoreDNS is running at https://127.0.0.1:50198/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Getting a URL response implies that kubectl is correctly configured to access your cluster.

Verify your kubectl installation:

$ kubectl get nodes
NAME       STATUS   ROLES                  AGE   VERSION
minikube   Ready    control-plane,master   64s   v1.23.3 
$ kubectl run helloworld --image=hello-world
pod/helloworld created
$ kubectl logs helloworld
Hello from Docker!
This message shows that your installation appears to be working correctly.
…………………………………………………………………………
For more examples and ideas, visit:
 https://docs.docker.com/get-started/
$ kubectl get pods
NAME         READY   STATUS             RESTARTS      AGE
helloworld   0/1     CrashLoopBackOff   2 (23s ago)   42s
$ minikube stop
✋  Stopping node "minikube"  ...
🛑  Powering off "minikube" via SSH ...
🛑  1 node stopped.

Workshop Outline

This workshop contains eleven modules to acquaint you with Kubernetes and its use in WMF’s production environments. The workshop introduces the following concepts:

• Executing a generic k8s cluster on Wikimedia Cloud Services (WMCS).
• Using WMF's deployment pipeline tools, namely Gerrit, Blubber, and Continuous Integration/Continuous Deployment (CI/CD) systems.

The appendix contains links to Docker tutorials that provide the necessary foundational knowledge. Each module should take around two or three hours to complete.

The workshop presents you with a hands-on coding experience, plenty of examples, and references to unblock you when you experience blockers.

Verify the following before attempting the workshop:

• Ensure you have Python running on your machines. Linux OS users can install Python 3 by running the following commands:

$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt update
$ sudo apt install python3
$ python3 --version
Python3.7.3

Your Python version output might be different from the above output. However, the output’s format remains the same.

• Most virtual machines come preinstalled with Git. However, you can install Git on your VM or configure your laptop’s terminal to run git commands. To verify whether you have GitHub installed on your VM, run the following command:

$ git version
-bash: Git: command not found

If you get the above output, you have to install Git; otherwise, you have Git installed on your VM. You can install Git by running the following commands:

$ sudo apt update
$ sudo apt install git-all

To verify your installation, run:

$ git version
git version 2.20.1

• Lastly, install pip on your VM:

$ sudo apt update
$ sudo apt install python3-pip
$ pip3 --version
pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)

• Run the hello-minikube application

Mac users on Big Sur should use this for the base setup.

Workshop Modules

Module 1: Set up a batch application on Kubernetes

Module 2: Build a service application on Kubernetes

Module 3: Setting up Infrastructure as Code (IaC) in Kubernetes

Module 4: Autoscaling in Kubernetes

Module 5: Refactoring code using Kubernetes features

Module 6: Building a production ready cluster

Module 7: Kubernetes Package Manager

Module 8: Manageability and Security

Module 9: Kubernetes at the Wikimedia Foundation (WMF)

Module 10: Using WMF Kubernetes to run your services

Add-on Module: Load Testing

Previous Work

A previous version of this workshop can be found here.

Additional Resources

This video by Cruise, presented at Kubecon, talks about the number of clusters and nodes. At the same time, this video highlights the achieved benefits. k8s is not the best solution for all environments, and this article gives some interesting points.