Kubernetes/Packages

Intro

For a variety of reasons, we deploy our kubernetes components in WMF production (Tools/Toolforge is a completely different environment) using Debian packages. Those are:

kubernetes-client one has kubectl in it, kubernetes-master has kube-apiserver, kube-scheduler, kube-controller-manager and kuberetes-node has kubelet and kube-proxy components.

Packaging

We usually build on the older debian version and copy the resulting packages to newer debian versions as needed.

We don't actually build kubernetes but package it's components from upstream binary releases as described above.

Part of the process is to download the release tarball and verify its sha512 hash against the one found in the current git master CHANGELOG.

Because of that, you will need to set HTTP proxy variables for internet access on the build host.

The general process to follow is:

Check out operations/debs/kubernetes on your workstation
Switch to the major version branch if it already exists or create a new one (vX.Y)
Create a patch to bump the debian changelog

Double-check that the environment variables DEBEMAIL and DEBFULLNAME are set correctly before running debchange (dch).

export NEW_VERSION=1.19.3 # Kubernetes version you want to package
dch -v ${NEW_VERSION}-1 -D bookworm-wikimedia "Update to v${NEW_VERSION}"
git commit debian/changelog

# Make sure to submit the patch to the correct branch
git review vX.Y

Merge

Check out operations/debs/kubernetes on the build host:

git clone "https://gerrit.wikimedia.org/r/operations/debs/kubernetes"

Build the packages:

git checkout vX.Y

# Ensure you allow networking in pbuilder
# This option needs to be in the file, an environment variable will *not* work!
echo "USENETWORK=yes" >> ~/.pbuilderrc

# Build the package
https_proxy=http://webproxy.$(hostname -d):8080 DIST=bookworm pdebuild

Publishing

When building a new Kubernetes minor version, you will have to add a new apt component first in modules/aptrepo/files/distributions-wikimedia

# On apt1002, copy the packages from the build host
# pbuilder-result is an rsync alias to /var/cache/pbuilder/result on the build host.
rsync -vaz build2001.codfw.wmnet::pbuilder-result/bookworm-amd64/kubernetes*<PACKAGE VERSION>* .

# Import packages to the corresponding kubernetes apt component
# components are listed at https://apt.wikimedia.org/wikimedia/dists/<version>/component
sudo -i reprepro -C component/kubernetesXY --ignore=wrongdistribution include bookworm-wikimedia /path/to/<PACKAGE>.changes

# Kubernetes packages can be copied between distros:
sudo -i reprepro -C component/kubernetesXY copysrc trixie-wikimedia bookworm-wikimedia kubernetes