You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Dragonfly

From Wikitech-static
Revision as of 09:43, 3 August 2021 by imported>JMeybohm
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Dragonfly is an P2P based file distribution system we use for distributing docker image layers to Kubernetes worker nodes.

Operations

Packaging

The code is hosted in operations/debs/dragonfly and uses Git-buildpackage flow.

Importing a new version

The imported upstream tarballs should include the complete vendor directory.

  • Check out the version (git tag) to import
$ ./debian/repack vX.Y.Z
  • This drops you into a shell with the git tag checked out. Do necessary changes here and commit
$ go mod vendor
$ git add -f vendor
# git diff --name-status --cached | grep -v 'vendor/' to make sure you only changed vendor
$ git commit -m "added vendor"
  • Exiting the shell will build a tarball to import
$ gbp import-orig /path/to/tarball.tar.xz
  • Push changes (including the tag crated by gpb) to gerrit
$ git push gerrit --all
$ git push gerrit --tags
  • Add a debian/changelog entry (as CR)
$ gbp dch
# Edit debian/changelog
$ git commit
$ git review

Building a new version

  • Check out the git repo on the build host
$ git clone "https://gerrit.wikimedia.org/r/operations/debs/dragonfly" && cd dragonfly
  • Build the package
$ BACKPORTS=yes gbp buildpackage --git-pbuilder --git-no-pbuilder-autoconf --git-dist=buster -sa -uc -us

Publish a new version

# on apt1001
rsync -vaz deneb.codfw.wmnet::pbuilder-result/buster-amd64/dragonfly* .
sudo -i reprepro -C main include buster-wikimedia /path/to/<PACKAGE>.changes

# Copy the package over to other distros (this is possible because they only contain static binaries)
sudo -i reprepro copysrc stretch-wikimedia buster-wikimedia dragonfly

Patches

If you need to add/update patches, please see: https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.patches.html

Resources