You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Help:Toolforge/Kubernetes"

From Wikitech-static
Jump to navigation Jump to search
imported>BryanDavis
(Add some out links to enwiki)
imported>MusikAnimal
(→‎Monitoring your jobs: how to kill your pod)
Line 58: Line 58:
* Runs the command <code>/data/project/stashbot/bin/stashbot.sh run</code> when the container starts.
* Runs the command <code>/data/project/stashbot/bin/stashbot.sh run</code> when the container starts.
* Mounts the <tt>/data/project/stashbot/</tt> NFS directory as <tt>/data/project/stashbot/</tt> inside the container.
* Mounts the <tt>/data/project/stashbot/</tt> NFS directory as <tt>/data/project/stashbot/</tt> inside the container.
=== Monitoring your jobs ===
You can see which jobs you have running with <code>kubectl get pods</code>. Using the name of the pod, you can see the logs with <code>kubectl logs <pod-name></code>.
To restart a failing pod, use <code>kubectl delete <pod-name></code>. If you need to kill it entirely, find the deployment name with <code>kubectl get deployment</code>, and delete it with <code>kubectl delete deployment <deployment-name></code>.


== Namespaces ==
== Namespaces ==
Line 101: Line 106:
== See also ==
== See also ==
* [[Portal:Toolforge/Admin/Kubernetes|Kubernetes administration]]
* [[Portal:Toolforge/Admin/Kubernetes|Kubernetes administration]]
* [https://kubernetes.io/docs/user-guide/kubectl-cheatsheet/ kubectl (Kubernetes command line tool) cheatsheet]

Revision as of 18:43, 5 November 2017

Kubernetes (often abbreviated k8s) is a platform for running containers. It is used in Toolforge to isolate Tools from each other and allow distributing Tools across a pool of servers.

Kubernetes webservices

The Toolforge webservice command has a --backend=kubernetes mode that will start, stop, and restart containers designed to run web services for various languages. See our Webservice help for more details.

Kubernetes cronjobs

It is possible to run cron jobs on kubernetes (see upstream documentation for a full description), however this functionality is not officially supported by the Wikimedia Cloud Services team and should be considered an experimental feature.

Kubernetes continuous jobs

The basic unit of managing execution on a Kubernetes cluster is called a "deployment". Each deployment is described with a YAML configuration file which describes the container images to be started ("pods" in the Kubernetes terminology) and commands to be run inside them after the container is initialized. A deployment also specifies where the pods run and what external resources are connected to them. The upstream documentation is comprehensive.

Example deployment.yaml

Stashbot is a Python irc bot that runs in a Kubernetes deployment. The deployment.yaml file that it uses to tell Kubernetes how to start the bot is reproduced below. This deployment is launched using a stashbot.sh wrapper script which runs kubectl create -f /data/project/stashbot/etc/deployment.yaml.

This deployment:

  • Uses the 'stashbot' namespace that the tool is authorized to control
  • Creates a container using the 'latest' version of the 'docker-registry.tools.wmflabs.org/toollabs-python2-base' Docker image.
  • Runs the command /data/project/stashbot/bin/stashbot.sh run when the container starts.
  • Mounts the /data/project/stashbot/ NFS directory as /data/project/stashbot/ inside the container.

Monitoring your jobs

You can see which jobs you have running with kubectl get pods. Using the name of the pod, you can see the logs with kubectl logs <pod-name>.

To restart a failing pod, use kubectl delete <pod-name>. If you need to kill it entirely, find the deployment name with kubectl get deployment, and delete it with kubectl delete deployment <deployment-name>.

Namespaces

Each tool has been granted control of a Kubernetes "namespace". Your tool can only create and control objects in its namespace. A tool's namespace is the same as the tool's name (e.g. admin, stashbot, hay, etc).

Container images

The Toolforge Kubernetes cluster is restricted to loading Docker images published at docker-registry.tools.wmflabs.org. These images are built using the Dockerfiles in the operations/docker-images/toollabs-images git repository.

Available container types

The webservice command has an optional type argument that allows you to choose which Docker container to run your Tool in.

Currently provided types:

  • golang
  • jdk8
  • nodejs
  • php5.6
  • python
  • python2
  • ruby2
  • tcl

FIXME: how does one get a list of the images?

A list of images is available in the overview page of the images source code repository for the Toolforge. Each folder name represents a Docker image.

php5.6 (Lighttpd + PHP)

Versions & Packages

This is running Debian Jessie, with PHP version 5.6. This is mostly compatible with PHP 5.5 which is present on the GridEngine Ubuntu Trusty nodes - so if your web service is currently running on Trusty with GridEngine, it should work on Kubernetes. Unlike on GridEngine, where all packages for all languages are installed and accessible, only PHP specific packages are available in this environment. The following packages are installed:

  • php5-cli
  • php5-curl
  • php5-gd
  • php5-imagick
  • php5-intl
  • php5-mcrypt
  • php5-mysqlnd
  • php5-pgsql
  • php5-redis
  • php5-sqlite
  • php5-xsl

This should match the php related packages installed on GridEngine exec nodes. Additional packages can be added on request.

See also