Portal:Toolforge/Admin/Kubernetes/Components

This page contains information on how to operate, build and deploy various Toolforge Kubernetes components.

The information in here is subject to change as we move in the direction of Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Kubernetes_component_workflow_improvements

In general, assume this is valid for both tools and toolsbeta.

Workflow

The workflow as we have it today.

Steps

This is what you normally would do:

code: make changes to a given code repository, push & merge as usual.
build: once code is ready to deploy to k8s, if the component needs it, build the container image using the cookbook.
pre-deploy: if the component uses a built container image, chances are that the image tag requires a refresh (avoid :latest). This is a new, separate commit, pointing to the newest built container image from previous step.
deploy: once ready to deploy to k8s for real, use the cookbook.
verify test by hand, verify, etc.
rollback if a rollback is needed, follow again the same steps but merging a revert commit plus (if required) an image tag bump.

Build

Use the wmcs.toolforge.k8s.component.build cookbook to build the docker container image that will later be deployed in the kubernetes cluster.

user@laptop:~$ cookbook wmcs.toolforge.k8s.component.build -h
usage: cookbooks.wmcs.toolforge.k8s.component.build [-h] [--project PROJECT] [--task-id TASK_ID] [--no-dologmsg]
                                                    [--registry-url REGISTRY_URL]
                                                    [--docker-builder-hostname DOCKER_BUILDER_HOSTNAME] --git-url GIT_URL
                                                    [--git-name GIT_NAME] [--git-branch GIT_BRANCH]
                                                    [--docker-image-tag DOCKER_IMAGE_TAG]
                                                    [--docker-image-name DOCKER_IMAGE_NAME]

WMCS Toolforge Kubernetes - build a docker image for a custom component

Usage example:
    cookbook wmcs.toolforge.k8s.component/build \
        --git-url https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api

options:
  -h, --help            show this help message and exit
  --project PROJECT     Relevant Cloud VPS openstack project (for operations, dologmsg, etc). If this cookbook is for hardware,
                        this only affects dologmsg calls. Default is 'tools'.
  --task-id TASK_ID     Id of the task related to this operation (ex. T123456). (default: None)
  --no-dologmsg         To disable dologmsg calls (no SAL messages on IRC). (default: False)
  --registry-url REGISTRY_URL
                        docker registry URL (default: docker-registry.tools.wmflabs.org)
  --docker-builder-hostname DOCKER_BUILDER_HOSTNAME
                        docker image builder virtual machine hostname (default: tools-docker-imagebuilder-01)
  --git-url GIT_URL     git URL for the source code (default: None)
  --git-name GIT_NAME   git repository name. If not provided, it will be guessed based on the git URL (default: None)
  --git-branch GIT_BRANCH
                        git branch in the source repository (default: main)
  --docker-image-tag DOCKER_IMAGE_TAG
                        docker tag for the new image, if not provided the git hash of the latest commit will be used (default:
                        None)
  --docker-image-name DOCKER_IMAGE_NAME
                        docker image name. If not provided, it will be guessed based on the git name (default: None)

Example:

user@laptop:~$ cookbook wmcs.toolforge.k8s.component.build --git-url https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api
[..]
user@laptop:~$ cookbook wmcs.toolforge.k8s.component.build --git-url https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer
INFO: guesses git tree name as jobs-framework-emailer
INFO: guesses docker image name as toolforge-jobs-framework-emailer
START - Cookbook wmcs.toolforge.k8s.component.build
INFO: using build node tools-docker-imagebuilder-01.tools.eqiad1.wikimedia.cloud
INFO: creating temp dir /tmp/cookbook-toolforge-k8s-component-build-ulprobicib
INFO: git cloning https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer
INFO: git checkout main on cloning /tmp/cookbook-toolforge-k8s-component-build-ulprobicib/jobs-framework-emailer
INFO: building docker image toolforge-jobs-framework-emailer:latest
INFO: cleaning up temp dir /tmp/cookbook-toolforge-k8s-component-build-ulprobicib
INFO: creating docker tag docker-registry.tools.wmflabs.org/toolforge-jobs-framework-emailer:latest
INFO: pushing to the registry docker-registry.tools.wmflabs.org/toolforge-jobs-framework-emailer:latest
[DOLOGMSG]: build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-emailer:latest from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer (084ee51)
END (PASS) - Cookbook wmcs.toolforge.k8s.component.build (exit_code=0)

Deploy

Use the wmcs.toolforge.k8s.component.deploy cookbook to deploy the custom component in the given kubernetes cluster.

user@laptop:~$ cookbook wmcs.toolforge.k8s.component.deploy -h
usage: cookbooks.wmcs.toolforge.k8s.component.deploy [-h] --cluster-name {tools,toolsbeta} [--task-id TASK_ID] [--no-dologmsg] --git-url GIT_URL
                                                     [--git-name GIT_NAME] [--git-branch GIT_BRANCH] [--deployment-command DEPLOYMENT_COMMAND]

WMCS Toolforge Kubernetes - deploy a kubernetes custom component

Usage example:
    cookbook wmcs.toolforge.k8s.component.deploy \
        --git-url https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api

options:
  -h, --help            show this help message and exit
  --cluster-name {tools,toolsbeta}
                        cluster to work on (default: None)
  --task-id TASK_ID     Id of the task related to this operation (ex. T123456). (default: None)
  --no-dologmsg         To disable dologmsg calls (no SAL messages on IRC). (default: False)
  --git-url GIT_URL     git URL for the source code (default: None)
  --git-name GIT_NAME   git repository name. If not provided, it will be guessed based on the git URL (default: None)
  --git-branch GIT_BRANCH
                        git branch in the source repository (default: main)
  --deployment-command DEPLOYMENT_COMMAND
                        command to trigger the deployment. (default: ./deploy.sh)

List

This is a list of our custom components and where to find them.

This is the only known source of truth for this information. Even if the list might drift outdated, as of this writing there is no other place in which this information is collected.

Third-party components

Additional consideration for some third-party components that we use.

Name	Deployment repository	Current version	K8s versions supported
Calico	https://gitlab.wikimedia.org/repos/cloud/toolforge/calico/	3.24.5	1.22 - 1.25^[1]
ingress-nginx	https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-nginx	1.4.0 (chart-4.3.0)	1.22 - 1.25^[2]
kube-state-metrics	https://gitlab.wikimedia.org/repos/cloud/toolforge/kubernetes-metrics	2.2.4 (chart-4.1.0)	?
metrics-server	https://gitlab.wikimedia.org/repos/cloud/toolforge/kubernetes-metrics	0.6.2 (chart-3.8.3)	1.19+^[3]
cadvisor	https://gitlab.wikimedia.org/repos/cloud/toolforge/kubernetes-metrics	0.30.2	n/a (not a Kubernetes specific tool)
Prometheus	https://gitlab.wikimedia.org/repos/cloud/toolforge/kubernetes-metrics	n/a
cert-manager	https://gitlab.wikimedia.org/repos/cloud/toolforge/cert-manager	1.11.0	1.21 - 1.26^[4]
reloader	https://gitlab.wikimedia.org/repos/cloud/toolforge/cert-manager	1.0.5	?
Tekton Pipelines	https://github.com/toolforge/buildservice/tree/main/deploy/base-tekton	0.33.2	1.20+^[5]

Portal:Toolforge/Admin/Kubernetes/Components

Contents

Workflow

Steps

Build

Deploy

List

Third-party components

References

See also

Navigation menu

Portal:Toolforge/Admin/Kubernetes/Components

Workflow

Steps

Build

Deploy

List

Third-party components

References

See also

Navigation menu

Search