You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Since Feb 2nd 2021, we use the upstream war file fetched from https://www.gerritcodereview.com/ but we still build plugins from sources. The source code is held in the Gerrit repository
- Install OpenJDK 8 and git (
gitpackages on Debian)
- Bazel is required (as of July 2020 at least bazel 3.4.1, can be checked in
Instead of installing Bazel directly, consider installing Bazelisk, which is a Bazel frontend that automatically installs the needed Bazel version for the Gerrit branch you are working on. Bazelisk saves you a lot of trouble when building Gerrit, so if you can, use it.
Update our repository
- Clone gerrit and submodules and add upstream as a remoteOur repository HEAD points to the branch holding the material for the deployment
git clone https://gerrit.wikimedia.org/r/operations/software/gerrit cd gerrit git remote add upstream https://gerrit.googlesource.com/gerrit
- Checkout the
stable-3.2branch (as of November 2020) that tracks the current upstream release:
git checkout stable-3.2 origin/stable-3.2
- Fetch the upstream branch and fast-forward our copy of their branch to the latest tag and push the
stable-3.2directly to Gerrit:
git fetch --tags upstream stable-3.2 git merge --ff-only v3.2.3 git push --tags origin HEAD:stable-3.2
wmf/stable-3.2and merge the latest tag, push the resulting SINGLE merge commit up for review:
git checkout wmf/stable-3.2 git merge v3.2.8
Next update git submodules. The
.gitmodules file comes from upstream and uses relative urls. Since your work copy points to our Gerrit, the submodules would be fetched from our Gerrit. Instead we have to instruct git to rewrite the URLs to use upstream repositories. This can be done using the git setting
url.<base>.insteadOf = <our url>:
- Update git submodules from the upstream repository:
git -c url."https://gerrit.googlesource.com".insteadOf="$(dirname $(git config --get remote.origin.url))" submodule update --init
- Push for review. At this point, you will likely be ahead of
wmf/stable-3.2by...quite a bit (hundreds of patches). This is all within a single merge commit though, so Be Bold — push up your changes:
git commit --ammend # to add Change-Id and Phabricator tracking bug git push origin HEAD:refs/for/wmf/stable-3.2
Fetch our additional plugins
As our Gerrit instance runs some plugins which are not part of Gerrit upstream set of plugins, we need to fetch them. Some plugins support out-of-tree builds, but the most stable approach is to simply clone them into directories underneath Gerrit's
plugins directory in a checkout of our
wmf/stable-3.2 branch. You can use the following short bash script (which assumes you can
ssh to our Gerrit and you have got
jq installed) to clone the needed repositories:
#!/bin/bash set -e set -o pipefail INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .')" DEPENDENCIES_FOR_BUILDING="its-base" cd plugins for PLUGIN in $INSTALLED_PLUGINS $DEPENDENCIES_FOR_BUILDING do if [ ! -e "$PLUGIN" ] then echo "Cloning $PLUGIN ..." git clone "https://gerrit.googlesource.com/plugins/$PLUGIN" fi done echo "Success. All needed plugins cloned."
Knowing which branch to choose for which repository is not so straight forward. If a plugin (most don't) provide version tags, use them if you are building Gerrit for a specific version. Otherwise, many plugins come with
stable- branches. Use the one for the Gerrit version you are building. If a plugin does not offer any of these, try the
Please go through each of the plugins individually to find the tag or branch that best fits the desired target. There are only 22 or so plugins, so while it is annoying, it will not take ages.
The following script go through each of our installed plugins, attempts to checkout the tag, else the stable branch and finally fallback to master. On completion it list for each plugin the git description (git describe --all) and the checked out branch.
#!/bin/bash set -eu -o pipefail if [ ! $# -eq 1 ]; then echo "Usage: $0 <gerrit git tag>" exit 2 fi TAG=$1 INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .')" echo "Checkout tag or branch or master..." for PLUGIN in $INSTALLED_PLUGINS; do ( cd "plugins/$PLUGIN" git checkout "$TAG" || git checkout origin/stable-3.2 || git checkout origin/master ); done; echo "Done." echo "Show the resolved branch or tag if HEAD is detached" for PLUGIN in $INSTALLED_PLUGINS; do ( cd "plugins/$PLUGIN" echo "-- $PLUGIN --" git describe --all git branch --points-at=HEAD ) done; echo "Done."
Fix-up Gerrit core submodules plugin
The plugins listed as submodules might be ahead of the tag/branch available and the heuristic above might have rolled them back in the past., for example when a plugin did not receive a tag. In this case, you should reinitialize the submodules:
git submodule update
Fix-up of gitiles plugin
gitiles plugin per default fetches fonts directly from Google. Since that's a third party dependency that we do not want, we instead want gitiles to use the fonts that Gerrit itself offers.
To achieve that, make sure your gitiles code includes change I4f4a0b7dd575cbc27641063e05d13b8a43a51d8b. The change did not get much traction upstream, so it probably won't land there. We nonetheless want to include that change.
Fix-up of javamelody plugin
javamelody plugin comes with external dependencies. Gerrit's build system does not automatically pick them up. To allow Gerrit to build the plugin, merge the contents of
plugins/javamelody/external_plugin_deps.bzl into Gerrit's own
Most plugins come with build instructions usually available at
As of Feb 2nd 2021 we no more build Gerrit ourselves, we instead use upstream .war.
- Checkout the
wmf/stable-3.2(as of November 2020)
git checkout -b wmf/stable-3.2 origin/wmf/stable-3.2
Production uses Java 8 as of November 19th 2020 which we will eventually switch to Java 11 ( https://phabricator.wikimedia.org/T268225 ). Meantime, Bazel has to be pointed to your local Java 8 installation:
JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 bazel build release
Note that if your building Gerrit v3.3 (or later) or master from 2020-10-06 (or later) and you're targeting Java 8, you'll also have to add the argument
--java_toolchain=//tools:error_prone_warnings_toolchain (this is untested, but claimed on the gerrit mailing list).
Since we use the upstream jar, we do not need to build the plugins listed as git submodules. The following script retrieve the list of our installed plugins, extract the bundled plugins from .gitmodules and remove them from the list of plugins to build.
#!/bin/bash set -eu set -o pipefail WAR_BUNDLED_PLUGINS=$(git config -f .gitmodules --get-regexp 'submodule\.plugins/.*\.path'|cut -d\ -f2|cut -d/ -f2) INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .')" TO_BUILD=$(/usr/bin/comm -13 <(echo "$WAR_BUNDLED_PLUGINS") <(echo "$INSTALLED_PLUGINS")) printf "== List of our extra plugins ==\n%s\n" "$TO_BUILD" for PLUGIN in $TO_BUILD do printf "=== Building %s ===\n" "$PLUGIN" JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 \ bazel build --workspace_status_command="python ./tools/workspace_status_release.py" plugins/"$PLUGIN" done;
Current Bazel version is 0.18.1, expected at least 0.19.0
- Bazel updates all the time, update your bazel version from the bazel website (ensure you verify the version signature with the [ https://bazel.build/bazel-release.pub.gpg bazel-release.pub.gpg] key. Or even better: Use Bazelisk, which is a transparent wrapper for Bazel that takes care of preparing the correct version for you.
curl: (35) error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
- bazel shells out to curl to download things and, sadly,
repo.eclipse.orgonly supports TLSv1
openssl s_client -connect repo.eclipse.org:443 -tls1. You can modify your openssl settings in
/etc/ssl/openssl.cnf. If your
MinProtocolis set to
TLSv1.2(because you're awesome) — change it to
TLSv1(because not everyone is awesome). Be sure to change it back when you're done.
- Copy the resulting
bazel-bin/plugins/$PLUGIN/$PLUGIN.jarto somewhere safe
- Checkout the
deploy/wmf/stable-Xbranch that aligns with the build branch
Copy in the
release.waryou saved as
- Copy all the plugin jars over to
./pluginsin the deploy branch
- Commit the changes, upload to Gerrit, etc
./upload_artifacts.py --version=2.15.2 gerrit.war plugins/foo.jar ...for the core application and all updated plugins (cf: Archiva, specifically setting up your
- Note: Alternatively, you can upload them by using the web UI, but that gets repetitive over a bunch of plugins so I wrote the tool above
- SSH to the deployment master
- Navigate to
- Fetch & checkout the appropriate
- Deploy to gerrit2001 to ensure that there are no errors with git-fat et al
scap deploy -l gerrit2001.wikimedia.org 'Gerrit to [version] on gerrit2001'
- Deploy to gerrit1001
scap deploy -l gerrit1001.wikimedia.org 'Gerrit to [version] on gerrit1001'
- If you're only deploying plugins, you're done, otherwise SSH to the Gerrit master and issue
sudo service gerrit restart