Jump to content

This is a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

APT repository

From Wikitech
Browse the repository contents at https://apt-browser.toolforge.org/

Wikimedia maintains its own APT repository available at https://apt.wikimedia.org/wikimedia/ and currently hosted on apt1002.wikimedia.org and apt2002.wikimedia.org.

This repository contains Debian and Ubuntu packages modified for use by Wikimedia as well as packages for Wikimedia-originated projects.

Repository Structure

We are using the following repository structure:

  • main contains all the packages that we either create internally or packages that we backport or modify from Debian and which are useful fleet-wide (e.g., Icinga plugins, Cumin, etc.)
  • thirdparty/hwraid contains binary-only drivers for RAID management used on baremetal servers only.
  • All other components using the thirdparty/ prefix are synchronised from external repositories (e.g., thirdparty/cloudera , thirdparty/confluent , thirdparty/ci (Jenkins) and thirdparty/k8s (Docker)). Please do not add internally built packages to thirdparty .
  • Some packages should not be available fleet-wide, but rather be used in more specific scenarios such as the following:
    • component/ci provides co-installable PHP packages that are only necessary for CI
    • "contain" specific packages that depend on outdated or backported libraries (e.g. crypto libraries)
    • ease migrations and upgrades for certain roles, while leaving the rest of the fleet untouched

You can find a complete list of repository components in our Puppet config

External Access

For Wikimedia servers and Cloud VPS instances, the repositories are automatically configured via Puppet .

To use this repository from an external host, the following lines need to be present in /etc/apt/sources.list or /etc/apt/sources.list.d/wikimedia.list : 

## Wikimedia APT repository
deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main

Use the right distribution, depending on which Debian/Ubuntu version was installed (e.g. bullseye-wikimedia , buster-wikimedia or trusty-wikimedia ).

Here is a complete list of all the available sources: 

deb http://apt.wikimedia.org/wikimedia trusty-wikimedia main experimental backports
deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main experimental backports
deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb http://apt.wikimedia.org/wikimedia buster-wikimedia main
deb http://apt.wikimedia.org/wikimedia bullseye-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports
deb-src http://apt.wikimedia.org/wikimedia trusty-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia buster-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia bullseye-wikimedia main

Additionally, to make sure that the system prefers packages from this repository, and not packages from the origin distributions even if they have a higher version number. For example, this APT source is "pinned" with a higher priority in /etc/apt/preferences.d/wikimedia.pref : 

Package: *
Pin: release o=Wikimedia
Pin-Priority: 1001

You can confirm Wikimedia's APT repository is taking preference now by running `apt-cache policy <package>`: 

$ apt-cache policy puppetmaster
puppetmaster:
  Installed: (none)
  Candidate: 4.8.2-5
  Version table:
     4.8.2-5 1001
        500 http://deb.debian.org/debian stretch/main amd64 Packages
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/main amd64 Packages

Security

The Wikimedia repository is signed with the Wikimedia Archive Automatic Signing Key <root@wikimedia.org> . This public key must be installed in APT's GPG keyring so it can verify packages successfully.

The key can be retrieved from /Stretch-Key (this key was introduced with stretch and also applies to later distros)

To install it, download the key to /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc : 

$ wget -O /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"

You can also retrieve the key from a Cloud VPS host: ssh <host> apt-key export root@wikimedia.org

See also

Retrieved from " https://wikitech.wikimedia.org/w/index.php?title=APT_repository&oldid=2316543 "