You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

User:Jeena Huneidi/Kubernetes Migration

From Wikitech-static
< User:Jeena Huneidi
Revision as of 05:49, 29 April 2020 by imported>Jeena Huneidi (add content)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Migrating a service to Kubernetes

A Guide With Examples From HelloWorldOid

TL;DR:

  1. Create .pipeline/blubber.yaml
  2. Generate dockerfile using blubber
  3. Create and test docker image
  4. Create .pipeline/config.yaml
  5. Update integration/config to run the pipeline you created for testing and publishing your service
  6. Create helm deployment chart
  7. Test in minikube (Try local-charts if you want to test integrations with other services/apps or do more development!)
  8. Run benchmarks and update deployment chart
  9. Talk to SRE about deployment to production

Set Up

We’re going to migrate your service to Kubernetes! If you have any questions, contact the Release_Engineering team.

Pre-requirements:

Clone the Repositories:

NOTE: Fetch and switch to the "kubernetes-tutorial" branch of helloworldoid for this guide.

Creating a Docker Image

Services running in production need a docker image generated and pushed to the wikimedia docker registry during CI. You'll need a .pipeline/blubber.yaml file like the one in the helloworldoid repository:

blubber.yaml:

helloworldoid blubber.yaml screenshot


blubber.yaml tells the blubber service what operating system, packages, libraries, and files are needed in your docker image. We need a docker image to deploy to Kubernetes because services in Kubernetes must be in a container. The blubber service will output a dockerfile that can be used to create your docker image. More detailed tutorials can be found here: https://wikitech.wikimedia.org/wiki/Blubber/Tutorial


1. Create your blubber.yaml file.

2. Use the blubberoid service to create your dockerfile from the blubber configuration! Switch to the root directory of your repo.

       curl -s "https://blubberoid.wikimedia.org/v1/production" \ 
               -H 'content-type: application/yaml' \
               --data-binary @".pipeline/blubber.yaml" > Dockerfile

3. Build the docker image: docker build -t <imagetag> -f - .

4. Test the docker image. For helloworldoid we don't need to supply any payload:

       docker run -d -p 8001:8001 <imagetag>
       curl localhost:8001

helloworldoid's response:

Screenshot of helloworldoid response


5. Clean up:

       docker ps
       docker stop <container id>
       docker rm <container id>

6. Commit your code and create a patchset. It will be needed in future steps.

Publishing Docker Images

It's great that our docker image runs, but we should take advantage of the continuous integration pipeline to build our images and publish them to a public repository so that others can use them too!

1. Switch over to the your repo's .pipeline folder. Create a config.yaml file like the one in helloworldoid:

config.yaml

helloworldoid config.yaml screenshot


config.yaml describes what actions need to happen in the continuous integration pipeline and what to publish, for example, tests and lint need to run before publishing a docker image. More detailed tutorials can be found here: https://wikitech.wikimedia.org/wiki/PipelineLib/Tutorial


2. Commit your config.yaml code and create a patchset.

3. Switch to the integration/config repo.

4. Edit jjb/project-pipelines.yaml:

project-pipelines.yaml

Create or edit pipelines and define jobs for your project, based on what you defined in your config.yaml. For example, helloworldoid has a test and a publish pipeline:


A screenshot of jjb/project-pipelines.yaml


5. Edit zuul/layout.yaml:

layout.yaml

Create or edit your repo's publish pipeline in the list of projects. Assign the trigger jobs defined in project-pipelines.yaml to the appropriate CI steps:


zuul/layout.yaml screenshot


6. Commit your changes and create a patchset.

Congratulations! After these changes are merged and deployed, your images will be published to docker-registry.wikimedia.org under the wikimedia namespace! The images in the registry can be seen here: https://tools.wmflabs.org/dockerregistry/

You can check here for more information about configuring CI: https://wikitech.wikimedia.org/wiki/PipelineLib/Guides/How_to_configure_CI_for_your_project

Our docker image has been built, but we still need a way to run it in Kubernetes.

Creating a Helm Chart

We use Helm charts to configure our Kubernetes deployments.

1. Switch to the deployment-charts repo.

2. Use the create_new_service.sh script to create our initial chart. Use the docker image from the wikimedia docker registry:


Screenshot of running the create_new_service script


3. Edit the files created by the script with specific configuration for our service. Let's take a look:

values.yaml

In the values.yaml for helloworldoid, I've edited two things - I've changed the default image tag to "stable", which is the tag my images is published with as defined in helloworldoid's blubber.yaml. I've also added the HELLO_WORLD environment variable, which helloworldoid expects to exist, as configurable:


values.yaml screenshot 1 values.yaml screenshot 2


Testing the Helm Chart

We can use helm commands to apply the chart and deploy our app to Minikube, but for this example, let's test that our chart works using the local-charts environment. If you want to test your app with other apps that have been migrated to Kubernetes, it might be easy to test it with local-charts. Add your new deployment-chart to local-charts:

1. In the local-charts repo, update helm/requirements.yaml, using the path to your deployment-charts chart as the repository:


requirements.yaml screenshot


2. Enable your service in values.yaml, and for testing purposes, disable any undesired services:


local-charts values.example.yaml screenshot 1


3. Try running your service in Kubernetes: From the root of the local-charts repo, type make deploy values=values.example.yaml in the terminal to deploy to Minikube.


helloworldoid deployment screenshot


4. now we can attempt a request to our running service:


helloworldoid response screenshot 1


Whoops, I forgot to add helloworldoid's configurables our values.example.yaml. I'll change it and run make update values=values.example.yaml to update our deployment.


local charts values.example.yaml screenshot 2 helloworldoid response screenshot 2


5. When everything is satisfactory switch to the deployment-charts repo's charts folder and use helm to package your chart:


helm package and index screenshot


6. Make sure to commit your changes in the deployment-charts repo and create a patchset. If you've added a new service to local-charts, why not also commit those changes and create a patchset for review?

Getting Deployed to Production

We have a deployment chart. What does it take to get our app deployed to production?

Running Benchmarks

Now that we know our HelloWorldOid runs in Kubernetes, we can run benchmarks to determine how many resources it needs. This is required for deployment to production.

1. Follow this tutorial to benchmark: https://wikitech.wikimedia.org/wiki/User:Alexandros_Kosiaris/Benchmarking_kubernetes_apps

2. Update the deployment-charts chart with the values discovered during the benchmark tests and push a patchset for review.

3. See https://wikitech.wikimedia.org/wiki/Deploying_a_service_in_kubernetes for more information, and then contact the serviceops team.