You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

GitLab/Gitlab Runner/Cloud Runners: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Ahmon Dancy
m (Made a guess at what word was intended)
imported>Jelto
mNo edit summary
 
Line 1: Line 1:
A set of instance-wide Shared Runners should offer CI capabilities for unreviewed (meaning untrusted) code for private projects and forks. This Runners should be available for every project by default and help the community and volunteers to have CI for every code change.
A set of instance-wide Shared Runners offer CI capabilities for unreviewed (meaning untrusted) code for private projects and forks. This Runners are available for every project by default and help the community and volunteers to have CI for every code change.
 
{{warning|1=Cloud Runners are work in progress. Some features may not be available.}}


=== Public Cloud Evaluation ===
=== Public Cloud Evaluation ===
Shared Cloud Runners execute unreviewed code for private projects and forks. This means Runners could be potentially compromised or run malicious code. The Runners should be ephemeral so any compromise can not persist and affect other jobs, especially not production jobs. To make sure resources are not abused, the Runners should be managed by a certain quota (CPU/Mem, execution time).
Shared Cloud Runners execute unreviewed code for private projects and forks. This means Runners could be potentially compromised or run malicious code. The Runners should be ephemeral so any compromise can not persist and affect other jobs, especially not production jobs. To make sure resources are not abused, the Runners should be managed by a certain quota (maximum CPU/Mem, execution timeout).


The above requirements could be offered by a public cloud provider such as AWS, GCP or Digital Ocean. These providers offer high elasticity, ephemeral machines, quotas and proper separation from production.
The above requirements could be offered by a public cloud provider such as AWS, GCP or Digital Ocean. These providers offer high elasticity, ephemeral machines, quotas and proper separation from production.


To support ephemeral, separated and resource-limited Runners the Docker or Kubernetes executor has to be used. The Kubernetes executor offers more flexibility regarding resource quotas and adding and removing ephemeral Runner hosts. So the goal should be to use a managed Kubernetes offering from a Cloud provider and use the Kubernetes executor. The benefit of using Kubernetes is also that migration to a different provider or WMCS should be easier. The integration to GitLab stays the same, only the provisioning of the underlying infrastructure changes.
To support ephemeral, separated and resource-limited Runners the Docker or Kubernetes executor have to be used. The Kubernetes executor offers more flexibility regarding resource quotas and adding and removing ephemeral Runner hosts. So the goal should be to use a managed Kubernetes offering from a Cloud provider and use the Kubernetes executor. The benefit of using Kubernetes is also that migration to a different provider or WMCS should be easier. The integration to GitLab stays the same, only the provisioning of the underlying infrastructure changes.
 
The cloud provider should also offer an interface to setup all components using code or API.
{| class="wikitable"
{| class="wikitable"
|+
|+
Line 41: Line 41:
|✔+ terraform provider
|✔+ terraform provider
|}
|}
The Runners do not perform critical CI/CD jobs, so any dependency to a public cloud provider is not affecting the ability to deploy production code. A termination of such a public cloud offering would disrupt private CI jobs mostly for the community and volunteers for a certain time.
 
===== API and automation =====
The cloud provider should also offer an interface to setup all components using code or API. The environment for Cloud Runners should be provisioned using APIs and/or infrastructure as code. Although the environment is expected to be very small with only one managed Kubernetes cluster this automation enables anyone to setup the environment. Furthermore it also helps as additional documentation (for example for migrating to a different environment or re-create the environment).
 
The industry standard tool for provisioning cloud resources is [https://www.terraform.io/ Terraform]. Terraform offers integrations for a wide range of Cloud providers, including the mentioned above. GitLab offers a [https://docs.gitlab.com/ee/user/infrastructure/iac/ Terraform integration] and management of a shared state. Different Cloud providers also have their own IoC tool, like Deployment Manager or CloudFormation. Using this provider-specific tooling makes it harder to switch between different environments. So Terraform should be favored.
 
===== Provider Dependency =====
The Runners do not perform critical CI/CD jobs, so any dependency to a public cloud provider is not affecting the ability to deploy production code. A termination of such a public cloud offering would disrupt private CI jobs from the community and volunteers. This feature of private CI is not available in Gerrit. So the risk would be to fall back to the same functionality Gerrit offers.
 
By using open source and known tools like Terraform and Kubernetes it is possible to migrate Cloud Runners to a different environment, if necessary.


=== Provisioning of Runner environment ===
=== Provisioning of Runner environment ===
The environment for Cloud Runners should be provisioned using APIs and/or infrastructure as code. Although the environment is expected to be very small with only one managed Kubernetes cluster this automation enables anyone to setup the environment. Furthermore it also helps as additional documentation (for example for migrating to a different environment or re-create the environment).
A dedicated GitLab project <code>[[gitlab:repos/releng/gitlab-cloud-runner|/repos/releng/cloud-runners]]</code> manages all of the Terraform code for provisioning for the environment. This project has a pipeline to execute the Terraform commands needed to create the cloud environment.  
 
The industry standard tool for provisioning cloud resources is [https://www.terraform.io/ Terraform]. Terraform offers IaC for a wide range of Cloud providers, including the mentioned above. GitLab offers a [https://docs.gitlab.com/ee/user/infrastructure/iac/ Terraform integration] and management of a shared state. Different Cloud providers also have their own IoC tool, like Deployment Manager or CloudFormation. Using this provider-specific tooling makes it harder to switch between different environments. So Terraform should be favored.


A dedicated GitLab project (<code>/releng/cloud-runners</code>) manages all of the Code and provisioning for the environment. This project has a pipeline to execute the Terraform commands needed to create the cloud environment.
The <code>README.md</code> contains further instructions for provisioning the Cloud Runners.


=== Integration of GitLab Runner into Kubernetes Cluster ===
=== Integration of GitLab Runner into Kubernetes Cluster ===
GitLab offers a dedicated [https://docs.gitlab.com/runner/executors/kubernetes.html Kubernetes Executor]. This executor interfaces with the Kubernetes API to create pods for CI jobs. The executor can be installed using the official [https://docs.gitlab.com/runner/install/kubernetes.html <code>gitlab/gitlab-runner</code> helm chart].
GitLab offers a dedicated [https://docs.gitlab.com/runner/executors/kubernetes.html Kubernetes Executor]. This executor interfaces with the Kubernetes API to create pods for CI jobs. The executor can be installed using the official [https://docs.gitlab.com/runner/install/kubernetes.html <code>gitlab/gitlab-runner</code> helm chart].


The helm chart could be applied by a dedicated CI job in a GitLab CI pipeline. (in <code>/releng/cloud-runners</code>).
The helm chart is applied by a dedicated CI job in a GitLab CI pipeline. (in [[gitlab:repos/releng/gitlab-cloud-runner|<code>/repos/releng/cloud-runners</code>]]).


=== Usage of Cloud Runners ===
=== Usage of Cloud Runners ===

Latest revision as of 15:32, 8 May 2022

A set of instance-wide Shared Runners offer CI capabilities for unreviewed (meaning untrusted) code for private projects and forks. This Runners are available for every project by default and help the community and volunteers to have CI for every code change.

Public Cloud Evaluation

Shared Cloud Runners execute unreviewed code for private projects and forks. This means Runners could be potentially compromised or run malicious code. The Runners should be ephemeral so any compromise can not persist and affect other jobs, especially not production jobs. To make sure resources are not abused, the Runners should be managed by a certain quota (maximum CPU/Mem, execution timeout).

The above requirements could be offered by a public cloud provider such as AWS, GCP or Digital Ocean. These providers offer high elasticity, ephemeral machines, quotas and proper separation from production.

To support ephemeral, separated and resource-limited Runners the Docker or Kubernetes executor have to be used. The Kubernetes executor offers more flexibility regarding resource quotas and adding and removing ephemeral Runner hosts. So the goal should be to use a managed Kubernetes offering from a Cloud provider and use the Kubernetes executor. The benefit of using Kubernetes is also that migration to a different provider or WMCS should be easier. The integration to GitLab stays the same, only the provisioning of the underlying infrastructure changes.

AWS GCP Digital Ocean
Access/

existing subscription

account exists,

security-team

? unknown account exists,

dedicated project for RelEng

Managed Kubernetes
Auto scaling nodepools
API/

Infrastructure as code

✔+ terraform provider ✔+ terraform provider ✔+ terraform provider
API and automation

The cloud provider should also offer an interface to setup all components using code or API. The environment for Cloud Runners should be provisioned using APIs and/or infrastructure as code. Although the environment is expected to be very small with only one managed Kubernetes cluster this automation enables anyone to setup the environment. Furthermore it also helps as additional documentation (for example for migrating to a different environment or re-create the environment).

The industry standard tool for provisioning cloud resources is Terraform. Terraform offers integrations for a wide range of Cloud providers, including the mentioned above. GitLab offers a Terraform integration and management of a shared state. Different Cloud providers also have their own IoC tool, like Deployment Manager or CloudFormation. Using this provider-specific tooling makes it harder to switch between different environments. So Terraform should be favored.

Provider Dependency

The Runners do not perform critical CI/CD jobs, so any dependency to a public cloud provider is not affecting the ability to deploy production code. A termination of such a public cloud offering would disrupt private CI jobs from the community and volunteers. This feature of private CI is not available in Gerrit. So the risk would be to fall back to the same functionality Gerrit offers.

By using open source and known tools like Terraform and Kubernetes it is possible to migrate Cloud Runners to a different environment, if necessary.

Provisioning of Runner environment

A dedicated GitLab project /repos/releng/cloud-runners manages all of the Terraform code for provisioning for the environment. This project has a pipeline to execute the Terraform commands needed to create the cloud environment.

The README.md contains further instructions for provisioning the Cloud Runners.

Integration of GitLab Runner into Kubernetes Cluster

GitLab offers a dedicated Kubernetes Executor. This executor interfaces with the Kubernetes API to create pods for CI jobs. The executor can be installed using the official gitlab/gitlab-runner helm chart.

The helm chart is applied by a dedicated CI job in a GitLab CI pipeline. (in /repos/releng/cloud-runners).

Usage of Cloud Runners