APT Staging repository

In addition to the main APT repository , which contains packages from upstream, we also run a repository for testing packages from Wikimedia-originated projects.

There is an import tool running in the APT Staging server that will automatically import packages built using the GitLab CI, if they match specific conditions (see below). Importing packages from Gerrit is currently not supported.

Requirements

To have your package imported automatically from Gitlab to the apt-staging repository, your package must meet the following conditions:

1. The package must be built using the build_ci_deb job in GitLab CI, part of the WMF DebCI pipeline ( Docs )
2. The GitLab project must be listed in the gitlab-trusted-runner repository.
3. The Git branch used by the CI job must be a protected branch (i.e., main, production, etc.)

Projects listed in the gitlab-trusted-runner repo are allowed to use Trusted Runners , but the import tool will only check that the project is listed in the git repo, not that the package was actually built on a Trusted Runner.

If you run the import tool manually, you can optionally bypass all of the above checks for development/testing purposes, but these should not be used in production to ensure that packages are built correctly and securely.

How the import tool works

A Systemd timer on the apt-staging host runs the import tool , which fetches the list of projects that are allowed to use trusted runners. Then, it fetches the last 50 CI jobs for those projects and checks to see if they match the requirements above.

If a package matches, it fetches it and places it in the repository "incoming" directory and then executes /usr/bin/reprepro -b /srv/aptrepo/wikimedia-staging processincoming default to pull the packages in to the repository.

Packages are signed with a separate key to the main apt repository, this is deployed with puppet and added to the root gpg keyring, then specified in the apt repo config .

Limitations

This is slightly more cumbersome because of the limitations with Gitlab. The GraphQL endpoint would allow us to perform a single search across all repos to find the packages that match our requirements, however this particular endpoint is not available in our plan.

Promote packages to the main repository

To promote packages to the main APT repository you need to set up the process with a puppet patch that updates the following two files.

In modules/aptrepo/files/updates , add an entry like this:

Name: vopsbot # Change to your package name
Method: https://apt-staging.wikimedia.org/wikimedia-staging/
Components: main # Change to the component you want to add it to
Suite: bookworm-wikimedia # If you have more than one suite, you will need to list them separately
UDebComponents:
Architectures: amd64
VerifyRelease: 671B51EA4CC7E5B2
ListShellHook: grep-dctrl -e -S '^vopsbot$' || [ $? -eq 1 ] # Change to your package name

In modules/aptrepo/files/distributions-wikimedia , add your package name to the relevant list given the Suite and Components that you have chosen.

Once you've done that, you can import your package in reprepro .

If you are unable to fetch the package, it might be due to a caching issue, see phab:T402284 for a workaround.

Access token for gitlab-package-puller script

The gitlab-package-puller script requires a read-api scoped access token for Gitlab for the regular sync. The token should be a group access token, with a "Guest" role, and the "read_api" scope. Token is created here: https://gitlab.wikimedia.org/groups/repos/-/settings/access_tokens .

The token is stored in private puppet in /private/modules/secret/secrets/apt-staging/gitlab-puller-token .

End-users don't need a dedicated access token and this section is mostly relevant to GitLab administrators.