You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

GitLab/Gitlab Runner/Trusted Runners: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Jelto
(add details about Trusted Runners)
imported>Jelto
(refactor requesting access and using Trusted Runners)
 
Line 1: Line 1:
Trusted GitLab Runners offer CI with additional security and trust requirements. In contrast to the [[GitLab/Gitlab Runner/Shared Runners|Shared Runners]], which run in WMCS, Trusted Runners live inside WMF infrastructure. With this approach, SRE team has full control over the instance and who has access. Furthermore customization, like scaling, other disks and NICs can be done outside of the bounds of WMCS. Beside that, [[GitLab/Gitlab Runner/Shared Runners|Shared Runners]] and Trusted Runners use the same puppet code (<code>role(gitlab_runner)</code>) with slightly different hiera configuration.
Trusted GitLab Runners offer CI with additional security and trust requirements. In contrast to the [[GitLab/Gitlab Runner/Shared Runners|Shared Runners]], which run in WMCS, Trusted Runners live inside WMF infrastructure. With this approach, SRE team has full control over the instance and who has access. Furthermore customization, like scaling, other disks and NICs can be done outside of the bounds of WMCS. Beside that, [[GitLab/Gitlab Runner/Shared Runners|Shared Runners]] and Trusted Runners use the same puppet code (<code>role(gitlab_runner)</code>) with slightly different hiera configuration.


The current Trusted Runner cluster consist of two Ganeti VMS:
The current Trusted Runner cluster consist of six physical machines:


* <code>gitlab-runner1001.eqiad.wmnet</code>  
* <code>gitlab-runner1002.eqiad.wmnet</code>
* <code>gitlab-runner2001.codfw.wmnet</code>
* <code>gitlab-runner1003.eqiad.wmnet</code>
 
* <code>gitlab-runner1004.eqiad.wmnet</code>
With increased usage this VMs may be replaced by hosts in the future.
* <code>gitlab-runner2002.codfw.wmnet</code>
* <code>gitlab-runner2003.codfw.wmnet</code>
* <code>gitlab-runner2004.codfw.wmnet</code>
See also [[GitLab/Gitlab Runner/Security Evaluation]] for all of the security measures enabled for Trusted Runners.


==== Request access to Trusted Runners ====
==== Request access to Trusted Runners ====
Line 12: Line 15:


Please make sure to check your project settings and especially who has maintainer permissions. You also must protect your main branch. As described in the [[GitLab/Gitlab Runner/Security Evaluation|Security Evaluation]] maintainer permissions ("merge", "+2") are needed to execute jobs on the Trusted Runners. Reversely this also means everyone with maintainer permissions can execute such jobs.
Please make sure to check your project settings and especially who has maintainer permissions. You also must protect your main branch. As described in the [[GitLab/Gitlab Runner/Security Evaluation|Security Evaluation]] maintainer permissions ("merge", "+2") are needed to execute jobs on the Trusted Runners. Reversely this also means everyone with maintainer permissions can execute such jobs.
Access to Trusted Runners is managed by https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner project. This is the root project where all Trusted Runners are registered. To request access, add your project to the <code>[[gitlab:repos/releng/gitlab-trusted-runner/-/blob/main/projects.json|projects.json]]</code> file and open a merge request. Please add the id (go to Settings > General in your project), the name of the project and a reason why access is needed.
This merge request will get reviewed and and the project will be added to Trusted Runners. Adding and removing projects is automated by a [[gitlab:repos/releng/gitlab-trusted-runner/-/blob/main/add-project.py|python script]] which uses the <code>[[gitlab:repos/releng/gitlab-trusted-runner/-/blob/main/projects.json|projects.json]]</code> .


==== Using Trusted Runners ====
==== Using Trusted Runners ====
To run a job on Trusted Runners, the project has to be authorized to use Trusted Runners (see above). Jobs can select runners based on tags. Currently Trusted Runners use the tag <code>protected</code>. So all jobs which need Trusted Runners should specify this tag:<syntaxhighlight lang="bash">
trusted-build-job:
  stage: trusted-build
  image: docker-registry.wikimedia.org/buster:20220109
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."
  tags:
    - protected # run job on protected / Trusted Runners
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
</syntaxhighlight>Please note Trusted Runners accept jobs from protected branches only. So make sure to protect the main branch and to trigger this job on protected branches only. Otherwise the jobs will get stuck.


==== Monitoring ====
==== Monitoring ====

Latest revision as of 12:49, 22 June 2022

Trusted GitLab Runners offer CI with additional security and trust requirements. In contrast to the Shared Runners, which run in WMCS, Trusted Runners live inside WMF infrastructure. With this approach, SRE team has full control over the instance and who has access. Furthermore customization, like scaling, other disks and NICs can be done outside of the bounds of WMCS. Beside that, Shared Runners and Trusted Runners use the same puppet code (role(gitlab_runner)) with slightly different hiera configuration.

The current Trusted Runner cluster consist of six physical machines:

  • gitlab-runner1002.eqiad.wmnet
  • gitlab-runner1003.eqiad.wmnet
  • gitlab-runner1004.eqiad.wmnet
  • gitlab-runner2002.codfw.wmnet
  • gitlab-runner2003.codfw.wmnet
  • gitlab-runner2004.codfw.wmnet

See also GitLab/Gitlab Runner/Security Evaluation for all of the security measures enabled for Trusted Runners.

Request access to Trusted Runners

Access to this Runners is gated and restricted. No project has access to Trusted Runners by default. Access has to be requested on project basis. Please use the following Phabricator task template to create a access request: Task Template

Please make sure to check your project settings and especially who has maintainer permissions. You also must protect your main branch. As described in the Security Evaluation maintainer permissions ("merge", "+2") are needed to execute jobs on the Trusted Runners. Reversely this also means everyone with maintainer permissions can execute such jobs.

Access to Trusted Runners is managed by https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner project. This is the root project where all Trusted Runners are registered. To request access, add your project to the projects.json file and open a merge request. Please add the id (go to Settings > General in your project), the name of the project and a reason why access is needed.

This merge request will get reviewed and and the project will be added to Trusted Runners. Adding and removing projects is automated by a python script which uses the projects.json .

Using Trusted Runners

To run a job on Trusted Runners, the project has to be authorized to use Trusted Runners (see above). Jobs can select runners based on tags. Currently Trusted Runners use the tag protected. So all jobs which need Trusted Runners should specify this tag:

trusted-build-job:
  stage: trusted-build
  image: docker-registry.wikimedia.org/buster:20220109
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."
  tags:
    - protected # run job on protected / Trusted Runners
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Please note Trusted Runners accept jobs from protected branches only. So make sure to protect the main branch and to trigger this job on protected branches only. Otherwise the jobs will get stuck.

Monitoring

Trusted Runners export Prometheus metrics. Dashboards are available in Grafana: gitlab-ci-overview and gitlab-runner-detail

Further Read

Related task: T295481