You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Docker-registry: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Legoktm
 
imported>Brennen Bearnes
mNo edit summary
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Kubernetes nav}}
We run our own Docker registry at [https://docker-registry.wikimedia.org docker-registry.wikimedia.org]. Internally the domain <code>docker-registry.discovery.wmnet</code> is also used. It is highly available (<code>docker_registry_ha</code> Puppet module) and backed by Swift. The registry is used by our k8s cluster, CI, and local development.
We run our own Docker registry at [https://docker-registry.wikimedia.org docker-registry.wikimedia.org]. Internally the domain <code>docker-registry.discovery.wmnet</code> is also used. It is highly available (<code>docker_registry_ha</code> Puppet module) and backed by Swift. The registry is used by our k8s cluster, CI, and local development.
The docker-registry nodes consist of the '''docker registry''' itself as well as an '''nginx reverse-proxy''' in front to handle '''authentication''' as well as '''local caching'''.


== Browsing ==
== Browsing ==
Line 6: Line 10:
== Downloading images ==
== Downloading images ==
Despite the name, the docker-registry is usable by any OCI container tool, including podman. Nearly all images may be publicly downloaded, examined, run, etc. The only exception is images under the <code>restricted/</code> namespace, which contain non-disclosed security patches and require specific credentials to fetch.
Despite the name, the docker-registry is usable by any OCI container tool, including podman. Nearly all images may be publicly downloaded, examined, run, etc. The only exception is images under the <code>restricted/</code> namespace, which contain non-disclosed security patches and require specific credentials to fetch.
Kubernetes nodes use [[Dragonfly]] to pull images.


== Uploading images ==
== Uploading images ==
Line 21: Line 27:
* <code>prod-build</code>: Can pull and push any non-restricted image. Used by deneb.codfw.wmnet via docker-pkg and build-base-images.
* <code>prod-build</code>: Can pull and push any non-restricted image. Used by deneb.codfw.wmnet via docker-pkg and build-base-images.
* <code>kubernetes</code>: Can pull any image (including "restricted/"). Used by k8s nodes to pull images, including the restricted MediaWiki production image.
* <code>kubernetes</code>: Can pull any image (including "restricted/"). Used by k8s nodes to pull images, including the restricted MediaWiki production image.
**See [[Kubernetes/Clusters/New#Access to restricted docker images]] for more details.


The passwords are all deployed using the private puppet repo. In case rotation is needed (e.g. compromise), grepping for <code><name>_user_password</code> should find all uses (switch hyphens to underscores).
The passwords are all deployed using the private puppet repo. In case rotation is needed (e.g. compromise), grepping for <code><name>_user_password</code> should find all uses (switch hyphens to underscores).


== Deleting images ==
== Deleting images ==
To delete an image entirely, you may use the tool <code>docker-registryctl</code> on the current build host. It will do it's best to remove the tags/image from the registry, despite the [[phab:T242604|circumstances]].
To delete an image entirely, you may use the tool <code>docker-registryctl</code> on the current build host. It will do it's best to remove the tags/image from the registry, despite the [[phab:T242604|circumstances]].<syntaxhighlight lang="bash">
elukey@deneb:~$ docker-registryctl delete-tags docker-registry.discovery.wmnet/istio/operator:1.6*
docker-registry.discovery.wmnet/istio/operator:1.6.14-1                  [DONE]
 
</syntaxhighlight>
 
== httpbb ==
There is a (not comprehensive) [[Httpbb]] test case for the docker registry:<syntaxhighlight lang="bash">
sudo httpbb /srv/deployment/httpbb-tests/docker-registry/test_docker-registry.yaml --hosts 'registry2008.codfw.wmnet'
</syntaxhighlight>


== See also ==
== See also ==
Line 33: Line 49:
* [[Docker]]
* [[Docker]]
* [https://github.com/distribution/distribution Upstream Git repository]
* [https://github.com/distribution/distribution Upstream Git repository]
*[[User:JMeybohm/Docker-Registry-Stresstest]]


[[Category:Containers]]
[[Category:Containers]]
[[Category:Docker]]

Latest revision as of 23:24, 10 May 2022

We run our own Docker registry at docker-registry.wikimedia.org. Internally the domain docker-registry.discovery.wmnet is also used. It is highly available (docker_registry_ha Puppet module) and backed by Swift. The registry is used by our k8s cluster, CI, and local development.

The docker-registry nodes consist of the docker registry itself as well as an nginx reverse-proxy in front to handle authentication as well as local caching.

Browsing

Visit https://docker-registry.wikimedia.org/ to see a list of images and their tags. The listing is updated on a hourly timer and is done by the registry-homepage-builder.py script in Puppet.

Downloading images

Despite the name, the docker-registry is usable by any OCI container tool, including podman. Nearly all images may be publicly downloaded, examined, run, etc. The only exception is images under the restricted/ namespace, which contain non-disclosed security patches and require specific credentials to fetch.

Kubernetes nodes use Dragonfly to pull images.

Uploading images

For services we recommend using the Deployment pipeline which is Blubber.

For other docker images, like infrastructure images, we manage them using docker-pkg, see: Kubernetes/Images#Image_building

Hosts that want to upload images must be individually listed in Puppet hiera.

Access control

The upstream docker-registry software provides no access control, so it is implemented at the nginx level, which restricts GET/POST/etc. requests accordingly. As of 2021-03-18, the following accounts exist:

  • ci-restricted: Can pull and push any image (including "restricted/"). Used by releases servers that build the restricted MediaWiki production image.
  • ci-build: Can pull and push any non-restricted image. Used by contint servers via docker-pkg and the deployment pipeline.
  • prod-build: Can pull and push any non-restricted image. Used by deneb.codfw.wmnet via docker-pkg and build-base-images.
  • kubernetes: Can pull any image (including "restricted/"). Used by k8s nodes to pull images, including the restricted MediaWiki production image.

The passwords are all deployed using the private puppet repo. In case rotation is needed (e.g. compromise), grepping for <name>_user_password should find all uses (switch hyphens to underscores).

Deleting images

To delete an image entirely, you may use the tool docker-registryctl on the current build host. It will do it's best to remove the tags/image from the registry, despite the circumstances.

elukey@deneb:~$ docker-registryctl delete-tags docker-registry.discovery.wmnet/istio/operator:1.6*
docker-registry.discovery.wmnet/istio/operator:1.6.14-1                   [DONE]

httpbb

There is a (not comprehensive) Httpbb test case for the docker registry:

sudo httpbb /srv/deployment/httpbb-tests/docker-registry/test_docker-registry.yaml --hosts 'registry2008.codfw.wmnet'

See also