ChartMuseum
ChartMuseum is an open-source Helm Chart Repository written in Go, with support for cloud storage backends like Swift .
It is used to store and serve Helm charts , which are packages of Kubernetes resources that are used to deploy apps to a Kubernetes cluster.
Interacting with ChartMuseum
Our stable repository is available at: https://helm-charts.wikimedia.org/stable/index.yaml (or as JSON: https://helm-charts.wikimedia.org/api/stable/charts )
For basic interaction with ChartMuseum, please see the
API docs
. For some tasks like packaging and uploading helm chart, there is
helm-chartctl
in the
python3-docker-report
package.
New charts/chart versions from operations/deployment-charts repository are packed and pushed to ChartMuseum every 2 minutes via systemd timers on the ChartMuseum nodes.
To add the stable repository to helm, use:
helm repo add wmf-stable https://helm-charts.wikimedia.org/stable/
Storage
ChartMuseum uses Swift as storage backend. It uses the "thanos-cluster", which is independent of Swift for media-storage and available at https://thanos-swift.discovery.wmnet . Data is replicated (without encryption) four times spanning codfw and eqiad (multi-region in Swift parlance) thus making the service fully multi-site.
If you need access to the storage directly, please see
Swift/How_To#Individual_Commands_-_interacting_with_Swift
. You will need to
source /etc/swift/account_AUTH_chartmuseum.env
and you will find the charts in the container:
charts
Operations
Pool/Depool
Chartmuseum uses
DNS/Discovery
(
dnsdisc=helm-charts
) and is active/active by default.
If you need to do some work, you may depool one of the sites using confctl:
confctl --object-type discovery select 'dnsdisc=helm-charts,name=codfw' set/pooled=false
Delete Charts
We're running Chartmuseum with deletes (via the API) disabled. If you need to remove particular charts/chart versions from the registry, you can do that via swift directly. Changes will be picked up by the Chartmuseum instances after some time (~ 60s).
SSH to a swift thanos cluster frontend (thanos-fe1004):
# Source the chartmuseum swift credentials
source /etc/swift/account_AUTH_chartmuseum.env
# To list all charts in the registry
swift list charts
# Delete one or many charts with
swift delete charts stable/cluster-0.1.2.tgz stable/cluster-0.1.3.tgz ... ...
Packaging
The code is hosted in operations/debs/chartmuseum and uses Git-buildpackage flow.
Importing a new version
The imported upstream tarballs should include the complete vendor directory.
- Check out the version (git tag) to import
$ ./debian/repack vX.Y.Z
- This drops you into a shell with the git tag checked out. Do necessary changes here and commit
$ go mod vendor $ git add -f vendor # git diff --name-status --cached | grep -v 'vendor/' to make sure you only changed vendor $ git commit -m "added vendor"
- Exiting the shell will build a tarball to import
$ gbp import-orig /path/to/tarball.tar.xz
- Push changes (including the tag crated by gpb) to gerrit
$ git push gerrit --all $ git push gerrit --tags
- Rebuild the example config file
$ go run generate_config_example.go > usr/share/doc/chartmuseum/examples/chartmuseum.yaml
- Add a debian/changelog entry (as CR)
$ gbp dch # Edit debian/changelog $ git commit $ git review
Building a new version
- Check out the git repo on the build host
- Build the package
$ BACKPORTS=yes WIKIMEDIA=yes gbp buildpackage --git-pbuilder --git-no-pbuilder-autoconf --git-dist=buster -sa -uc -us
Patches
If you need to add/update patches, please see: https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.patches.html