Jump to content

This is a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Kubernetes/Kubectl

From Wikitech

kubectl is the Kubernetes command-line tool to deploy and manage applications on a Kubernetes cluster.

For tasks like doing deployments kubectl should not be used. Instead please refer to helm and helmfile .

On Debian 9 (also known as Debian Stretch ) it is included in the package kubernetes-client .

kubectl can be used to view and troubleshoot cluster components and to do certain maintenance tasks. See also the kubectl cheatsheet/ .

Run kubectl from the deploy servers

Setting KUBECONFIG

kubectl needs to know how to connect to the cluster and what credentials to use. This information can be stored in the kubeconfig file. By default kubectl uses a variable called KUBECONFIG to find this kubeconfig file. If the variable is not set, it looks in ~/.kube/config . [ 1 ]

In order to have kubectl working with your service the KUBECONFIG variable has to be set. This is of the form:

KUBECONFIG="/etc/kubernetes/${SERVICE}-${CLUSTER}.config"

It is easier to use kube_env for this purpose, see below.

Running kubectl Commands

For example to get a list of pods for a service:

KUBECONFIG="/etc/kubernetes/termbox-staging.config" kubectl get pods

Using kube_env

The tool kube_env (present on deploy hosts) simplifies the configuration of kubectl and the access to different clusters and services.

tl;dr

Use the following command to setup the environment to execute kubectl commands in the desired namespace (by convention named the same as the service):

kube_env $NAMESPACE $CLUSTER

Then you can use kubectl to access the resources in the service's namespace:

kubectl get pods

kube_env is aliased to kube-env for consistency with usual CLI tools

kube-environments and kube-services will list for you the available environments and namespaces, which you can also access through the $KUBERNETES_ENV_ENVIRONMENTS and $KUBERNETES_ENV_SERVICES env variables.

What kube_env actually does

kube_env sets up the KUBECONFIG environment variable to point to the right file for the desired service and user. By convention, for each service, we create an eponymous namespace, as well as two users with varying privilege. For example, for mw-debug in the staging cluster:

cgoubert@deploy2002:~$ ls -1 /etc/kubernetes/mw-debug*staging.config
# KUBECONFIG for the mw-debug-deploy user
/etc/kubernetes/mw-debug-deploy-staging.config

# KUBECONFIG for the mw-debug user
/etc/kubernetes/mw-debug-staging.config

cgoubert@deploy2002:~$ kube_env mw-debug staging
cgoubert@deploy2002:~$ echo $KUBECONFIG
/etc/kubernetes/mw-debug-staging.config

cgoubert@deploy2002:~$ kube_env mw-debug-deploy staging
cgoubert@deploy2002:~$ echo $KUBECONFIG
/etc/kubernetes/mw-debug-deploy-staging.config

We abuse this naming convention and tell users that the first argument to kube_env is the service or the namespace, when it is actually the user.

Privileged user (for admin access)

The "usual" users have restricted privileges. If more access is needed, root (on deploy hosts) has access to the admin user:

user@deploy1002:~$ sudo -i
root@deploy1002:~# kube_env admin $CLUSTER

There is also a slightly more privileged user than the default available to deployers:

user@deploy1002:~$ kube_env $NAMESPACE-deploy $CLUSTER

Describe a pod

If there is an issue with a pod not starting and logs don't say anything, use "describe" to get more info. For example:

kubectl describe pod P -n sessionstore

Also see

  1. https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/#set-the-kubeconfig-environment-variable