You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Continuous Delivery

From Wikitech-static
Revision as of 19:17, 15 October 2018 by imported>Lars Wirzenius (typo fix)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Conceptual Overview of Continuous Delivery Pipeline

Wikimedia's Continuous Delivery pipeline for Streamlined Service Delivery is a project that is still in its infancy as of Fall 2018. The idea is to use a set of configuration files residing in the .pipeline directory of a project to test, verify, build, and release production Docker images of a service and then deploy those services to the production Kubernetes cluster. Currently, the Mathoid repository is deployed to Kubernetes in production via the Continuous Delivery pipeline.

Design

For design details, refer to Streamlined Service Delivery Design

Jobs

Continuous Delivery jobs are run through our Jenkins installation. There are two jobs that are run for each repository:

The Service Pipeline Test Only job is triggered when a patch is submitted for review to a supported Gerrit repository by a trusted user. The Service Pipeline job is triggered when a patch is merged into a supported Gerrit repository. The Service Pipeline job is a superset of the tasks run by Service Pipeline Test Only

The job definitions for both of these jobs live in our integration/config repository on Gerrit. Both utilize code from the integration/pipelinelib library written by Dan Duvall.

Service Pipeline

The Service Pipeline job is triggered when a patch is merged into a supported Gerrit repository. It performs the following tasks:

  1. Looks for Blubber configuration file in ./.pipeline/blubber.yaml
  2. Builds the test Blubber variant
  3. Runs the entrypoint defined by the test Blubber variant
  4. Builds the production Blubber variant
  5. Looks for the current Helm chart version in ./.pipeline/helm.yaml
  6. Uses the defined Helm chart to helm deploy the newly built production Blubber variant to the CI namespace of the production Kubernetes cluster
  7. Runs helm test against the deployment (in practice, currently, this means service-checker is run against the deployment)
  8. If all that passes, the production container is pushed to the production Docker image repository at https://docker-registry.wikimedia.org

Service Pipeline Test Only

The Service Pipeline Test Only job is a subset of the checks performed by the full Service Pipeline. Specifically, it builds and runs the test Blubber variant when a patch is submitted for review to a supported Gerrit repository by a trusted user.