You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Phabricator project: #Gerrit

Gerrit/Upgrade: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Hashar
(Created page with " Since Feb 2nd 2021, we use the upstream war file fetched from https://www.gerritcodereview.com/ but we still build plugins from sources. The source code is held in the Gerrit...")
 
imported>Hashar
({{gerrit nav}})
Line 1: Line 1:
{{gerrit nav}}
Since Feb 2nd 2021, we deploy the upstream war file as is. It is fetched from https://www.gerritcodereview.com/ . We still build extra plugins from sources though using the source code held in the Gerrit repository {{git repo|operations/software/gerrit}}.


Since Feb 2nd 2021, we use the upstream war file fetched from https://www.gerritcodereview.com/ but we still build plugins from sources. The source code is held in the Gerrit repository <code>operations/software/gerrit</code>.
The upstream branches (<code>master</code>, <code>stable-3.2</code>) are manually kept in sync. We have a forked branch <code>wmf/stable-3.2</code> which has our extra plugins registered as sub-modules as well as some Wikimedia specific build scripts at the root of the repository.  


== Prerequisites ==
== Build prerequisites ==
# Install OpenJDK 8 and git (<code>openjdk-8-jdk</code> and <code>git</code> packages on Debian)
Install OpenJDK 8, git, NodeJS, Python 3. On Debian:<syntaxhighlight lang="console">
# [https://docs.bazel.build/versions/master/install.html Bazel] is required (as of July 2020 at least bazel 3.4.1, can be checked in<code>.bazelversion</code> ).
$ sudo apt install git openjdk-8-jdk-headless nodejs python3
Instead of installing Bazel directly, consider installing [https://github.com/bazelbuild/bazelisk Bazelisk], which is a Bazel frontend that automatically installs the needed Bazel version for the Gerrit branch you are working on. Bazelisk saves you a lot of trouble when building Gerrit, so if you can, use it.
</syntaxhighlight>Gerrit and its plugins are build using [https://docs.bazel.build/versions/master/install.html Bazel]. Since the exact version to use might differ between branches or can be updated by upstream at any point, one should install [https://github.com/bazelbuild/bazelisk Bazelisk]. It is a frontend to Bazel which automatically installs the needed Bazel version for the Gerrit branch you are working on. You will require the Go programming language in order to build it. The following will build Bazelisk 1.6.1 from source and copy the resulting binary in your user bin directory (<code>~/.local/bin</code>):<syntaxhighlight lang="console">
$ which go || sudo apt install golang-go
$ mkdir /tmp/bazelisk
$ (/tmp/bazelisk-build && go mod init . && go get github.com/bazelbuild/bazelisk@v1.6.1)
$ mkdir -p $HOME/.local/bin && cp /tmp/bazelisk-build/gopath/bin/bazelisk $HOME/.local/bin/bazelisk
$ rm -fR /tmp/bazelisk-build
</syntaxhighlight>


== Update our repository ==
== Update our repository ==
# Clone gerrit and submodules and add upstream as a remote<syntaxhighlight lang="bash">
 
=== Upgrade Gerrit core branch ===
#Clone Gerrit and submodules from Wikimedia add upstream as a remote<syntaxhighlight lang="bash">
git clone https://gerrit.wikimedia.org/r/operations/software/gerrit
git clone https://gerrit.wikimedia.org/r/operations/software/gerrit
cd gerrit
cd gerrit
git remote add upstream https://gerrit.googlesource.com/gerrit
git remote add upstream https://gerrit.googlesource.com/gerrit
</syntaxhighlight>Our repository HEAD points to the branch holding the material for the deployment <code>deploy/wmf/stable-3.2</code>
</syntaxhighlight>Our repository <code>HEAD</code> points to the branch holding the material for the deployment <code>deploy/wmf/stable-3.2</code>
#Checkout the <code>stable-3.2</code> branch (as of November 2020) that tracks the current upstream release:<syntaxhighlight lang="bash">
#Checkout the <code>stable-3.2</code> branch (as of November 2020) which is a copy of the upstream release we currently use:<syntaxhighlight lang="bash">
git checkout stable-3.2 origin/stable-3.2
git checkout stable-3.2 origin/stable-3.2
</syntaxhighlight>
</syntaxhighlight>
# Fetch the upstream branch and fast-forward our copy of their branch to the latest tag and push the <code>stable-3.2</code> directly to Gerrit:<syntaxhighlight lang="bash">
# Fetch the upstream branch and fast-forward our copy of their branch to the latest tag. Then push the <code>stable-3.2</code> directly to Gerrit:<syntaxhighlight lang="bash">
git fetch --tags upstream stable-3.2
git fetch --tags upstream stable-3.2
git merge --ff-only v3.2.3
git merge --ff-only v3.2.3
Line 25: Line 35:
git merge v3.2.8
git merge v3.2.8
</syntaxhighlight>
</syntaxhighlight>
Next update git submodules. The <code>.gitmodules</code> file comes from upstream and uses '''relative urls'''. Since your work copy points to our Gerrit, the submodules would be fetched from our Gerrit. Instead we have to instruct git to rewrite the URLs to use upstream repositories. This can be done using the git setting <code>url.<base>.insteadOf = <our url></code>:
 
=== Update bundled submodules ===
Gerrit core comes with bundled plugins and a specific version of jgit which are registered as submodules. The <code>.gitmodules</code> file comes from upstream and uses '''relative urls'''. Since your work copy points to the Wikimedia Gerrit, the submodules would be fetched from our Gerrit which will fail since the repositories do not exist. Instead we have to instruct git to rewrite the URLs to use upstream repositories. This can be done using the git configuration setting <code>url.<base>.insteadOf = <our url></code>.
 
#Ensure you are in our forked branch:<syntaxhighlight lang="bash">
git checkout wmf/stable-3.2
# update as needed
</syntaxhighlight>
#Update git submodules from the upstream repository: <syntaxhighlight lang="bash">
#Update git submodules from the upstream repository: <syntaxhighlight lang="bash">
git -c url."https://gerrit.googlesource.com".insteadOf="$(dirname $(git config --get remote.origin.url))" submodule update --init
git -c url."https://gerrit.googlesource.com".insteadOf="$(dirname $(git config --get remote.origin.url))" submodule update --init
Line 36: Line 53:


== Fetch our additional plugins ==
== Fetch our additional plugins ==
As our Gerrit instance runs some plugins which are not part of Gerrit upstream set of plugins, we need to fetch them. Some plugins support out-of-tree builds, but the most stable approach is to simply clone them into directories underneath Gerrit's <code>plugins</code> directory in a checkout of our <code>wmf/stable-3.2</code> branch. You can use the following short bash script (which assumes you can <code>ssh</code> to our Gerrit and you have got <code>jq</code> installed) to clone the needed repositories:
As our Gerrit instance runs some plugins which are not part of Gerrit upstream set of plugins, we have those extra plugins registered and they need to be updated to match the new Gerrit core. You will need <code>ssh</code> admin access to Gerrit to be able to list the installed plugins as well as <code>jq</code> (<code>sudo apt install jq</code>).
<syntaxhighlight lang="bash">
List our plugins:<syntaxhighlight lang="console">
#!/bin/bash
$ ./wmf-plugins-list.sh
 
plugins/go-import
set -e
plugins/healthcheck
set -o pipefail
plugins/its-base
 
plugins/its-phabricator
INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .[]')"
plugins/javamelody
DEPENDENCIES_FOR_BUILDING="its-base"
plugins/lfs
 
plugins/metrics-reporter-jmx
cd plugins
plugins/metrics-reporter-prometheus
 
plugins/motd
for PLUGIN in $INSTALLED_PLUGINS $DEPENDENCIES_FOR_BUILDING
plugins/reviewers
do
plugins/zuul
  if [ ! -e "$PLUGIN" ]
$  
  then
    echo "Cloning $PLUGIN ..."
    git clone "https://gerrit.googlesource.com/plugins/$PLUGIN"
  fi
done
 
echo "Success. All needed plugins cloned."
</syntaxhighlight>
</syntaxhighlight>




 
Most plugins come with a set of <code>stable-</code> branches, some support multiple versions and solely have a <code>master</code> branch. The proper branch to use for our extra plugins is indicated in  <code>.gitmodules</code> , we thus just have to update them from the remote branch (<code>git submodule update --remote plugins/foo</code>). To update our plugins:<syntaxhighlight lang="console">
Knowing which branch to choose for which repository is not so straight forward. If a plugin (most don't) provide version tags, use them if you are building Gerrit for a specific version. Otherwise, many plugins come with <code>stable-</code> branches. Use the one for the Gerrit version you are building. If a plugin does not offer any of these, try the <code>master</code> branch.
$ ./wmf-plugins-update.sh
 
Updating Wikimedia specific plugins
Please go through each of the plugins individually to find the tag or branch that best fits the desired target. There are only 22 or so plugins, so while it is annoying, it will not take ages.
<the list of our plugins is shown here>
 
<git submodule update if relevant>
The following script go through each of our installed plugins, attempts to checkout the tag, else the stable branch and finally fallback to master. On completion it list for each plugin the git description (git describe --all) and the checked out branch.
Done
<syntaxhighlight lang="bash">
$
#!/bin/bash
 
set -eu -o pipefail
 
if [ ! $# -eq 1 ]; then
  echo "Usage: $0 <gerrit git tag>"
  exit 2
fi
 
TAG=$1
INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .[]')"
 
echo "Checkout tag or branch or master..."
for PLUGIN in $INSTALLED_PLUGINS; do
  (
    cd "plugins/$PLUGIN"
    git checkout "$TAG" || git checkout origin/stable-3.2 || git checkout origin/master
  );
done;
echo "Done."
 
echo "Show the resolved branch or tag if HEAD is detached"
for PLUGIN in $INSTALLED_PLUGINS; do
  (
    cd "plugins/$PLUGIN"
    echo "-- $PLUGIN --"
    git describe --all
    git branch --points-at=HEAD
    )
done;
echo "Done."
</syntaxhighlight>
 
=== Fix-up Gerrit core submodules plugin ===
The plugins listed as submodules might be ahead of the tag/branch available and the heuristic above might have rolled them back in the past., for example when a plugin did not receive a tag.  In this case, you should reinitialize the submodules:<syntaxhighlight lang="bash">
git submodule update
</syntaxhighlight>
</syntaxhighlight>


=== Fix-up of gitiles plugin ===
=== Fix-up of gitiles plugin ===
'''''TODO we should fork it.'''''


The <code>gitiles</code> plugin per default fetches fonts directly from Google. Since that's a third party dependency that we do not want, we instead want gitiles to use the fonts that Gerrit itself offers.
The <code>gitiles</code> plugin per default fetches fonts directly from Google. Since that's a third party dependency that we do not want, we instead want gitiles to use the fonts that Gerrit itself offers.
Line 113: Line 88:


=== Fix-up of javamelody plugin ===
=== Fix-up of javamelody plugin ===
'''''TODO should not be needed'''''


The <code>javamelody</code> plugin comes with external dependencies. Gerrit's build system does not automatically pick them up. To allow Gerrit to build the plugin, merge the contents of <code>plugins/javamelody/external_plugin_deps.bzl</code> into Gerrit's own <code>plugins/external_plugin_deps.bzl</code>
The <code>javamelody</code> plugin comes with external dependencies. Gerrit's build system does not automatically pick them up. To allow Gerrit to build the plugin, merge the contents of <code>plugins/javamelody/external_plugin_deps.bzl</code> into Gerrit's own <code>plugins/external_plugin_deps.bzl</code>




Most plugins come with build instructions usually available at <code>src/main/resources/Documentation/build.html</code>.
Once done updating our plugins submodules, git add them, update your commit and send it for review:<syntaxhighlight lang="bash">
git add $(./wmf-plugins-list.sh)
git commit --amend
git push origin HEAD/refs/for/wmf/stable-3.2
</syntaxhighlight>


== Build Gerrit ==
== Build Gerrit ==
Line 136: Line 116:


=== Plugins ===
=== Plugins ===
Since we use the upstream jar, we do not need to build the plugins listed as git submodules. The following script retrieve the list of our installed plugins, extract the bundled plugins from .gitmodules and remove them from the list of plugins to build.<syntaxhighlight lang="bash">
Most plugins come with build instructions usually available at <code>src/main/resources/Documentation/build.html</code>.
#!/bin/bash
 
Since we use the upstream war, we do not need to build the plugins provided with Gerrit core. Instead we only build our extra plugins. The <code>wmf-plugins-build.sh</code> script iterate over the plugins found via <code>wmf-plugins-list.sh</code> and build them, taking care of dependencies files if needed:<syntaxhighlight lang="console">
$ ./wmf-plugins-list.sh
plugins/go-import
plugins/healthcheck
plugins/its-base
plugins/its-phabricator
plugins/javamelody
plugins/lfs
plugins/metrics-reporter-jmx
plugins/metrics-reporter-prometheus
plugins/motd
plugins/reviewers
plugins/zuul
$
</syntaxhighlight>
Build them:<syntaxhighlight lang="console">
$ ./wmf-plugins-build.sh
Building Wikimedia specific plugins


set -eu
> Building go-import
set -o pipefail


WAR_BUNDLED_PLUGINS=$(git config -f .gitmodules --get-regexp 'submodule\.plugins/.*\.path'|cut -d\  -f2|cut -d/ -f2)
...
INSTALLED_PLUGINS="$(ssh -p 29418 gerrit.wikimedia.org gerrit plugin ls --format JSON | jq --raw-output 'keys | .[]')"


TO_BUILD=$(/usr/bin/comm -13 <(echo "$WAR_BUNDLED_PLUGINS") <(echo "$INSTALLED_PLUGINS"))
Done building
printf "== List of our extra plugins ==\n%s\n" "$TO_BUILD"
Copying build artifacts
plugins-wmf/go-import.jar
plugins-wmf/healthcheck.jar
plugins-wmf/its-base.jar
plugins-wmf/its-phabricator.jar
plugins-wmf/javamelody.jar
plugins-wmf/lfs.jar
plugins-wmf/metrics-reporter-jmx.jar
plugins-wmf/metrics-reporter-prometheus.jar
plugins-wmf/motd.jar
plugins-wmf/reviewers.jar
plugins-wmf/zuul.jar


Those should be copied to the wmf/deploy branch


for PLUGIN in $TO_BUILD
$  
do
  printf "=== Building %s ===\n" "$PLUGIN"
  JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 \
    bazel build --workspace_status_command="python ./tools/workspace_status_release.py" plugins/"$PLUGIN"
done;
</syntaxhighlight>
</syntaxhighlight>
=== Troubleshooting ===
;<code>Current Bazel version is 0.18.1, expected at least 0.19.0</code> : Bazel updates all the time, update your bazel version from [https://www.bazel.build/ the bazel website] (ensure you verify the version signature with the [ https://bazel.build/bazel-release.pub.gpg bazel-release.pub.gpg] key. Or even better: Use [https://github.com/bazelbuild/bazelisk Bazelisk], which is a transparent wrapper for Bazel that takes care of preparing the correct version for you.
; <code>curl: (35) error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol</code> : bazel shells out to curl to download things and, sadly, <code>repo.eclipse.org</code> only supports TLSv1 <code>openssl s_client -connect repo.eclipse.org:443 -tls1</code>. You can modify your openssl settings in <code>/etc/ssl/openssl.cnf</code>. If your <code>MinProtocol</code> is set to <code>TLSv1.2</code> (because you're awesome) — change it to <code>TLSv1</code> (because not everyone is awesome). Be sure to change it back when you're done.


== Upload artifacts ==
== Upload artifacts ==
# Copy the resulting <s><code>bazel-bin/release.war</code> and</s> <code>bazel-bin/plugins/$PLUGIN/$PLUGIN.jar</code> to somewhere safe
# Checkout the <code>deploy/wmf/stable-X</code> branch that aligns with the build branch
# Checkout the <code>deploy/wmf/stable-X</code> branch that aligns with the build branch
# <s>Copy in the <code>release.war</code> you saved as <code>./gerrit.war</code></s>
# Copy all the plugin jars from <code>./plugins-wmf</code>over to <code>./plugins</code> in the deploy branch
# Copy all the plugin jars over to <code>./plugins</code> in the deploy branch
#Ensure Gerrit bundled plugins are not included and that any plugins we no more have installed are removed
# Commit the changes, upload to Gerrit, etc
# Commit the changes and upload to Gerrit
# Run <code>./upload_artifacts.py --version=2.15.2 gerrit.war plugins/foo.jar ...</code> for the core application and all updated plugins (cf: [[Archiva]], specifically setting up your <code>~/.m2/settings.xml</code>
# Run <code>./upload_artifacts.py --version=3.2.8 gerrit.war plugins/foo.jar ...</code> for the core application and all updated plugins (cf: [[Archiva]], specifically setting up your <code>~/.m2/settings.xml</code>
#: '''Note''': Alternatively, you can upload them by using the web UI, but that gets repetitive over a bunch of plugins so I wrote the tool above
#: '''Note''': Alternatively, you can upload them by using the web UI, but that gets repetitive over a bunch of plugins hence why we wrote the above tool.


== Deploying ==
== Deploying ==
# SSH to the deployment master
# SSH to the deployment master (<code>ssh deployment.eqiad.wmnet</code>)
# Navigate to <code>/srv/deployment/gerrit/gerrit</code>
# Navigate to <code>/srv/deployment/gerrit/gerrit</code>
# Fetch & checkout the appropriate <code>deploy/wmf/stable-X</code> branch
# Fetch & checkout the appropriate <code>deploy/wmf/stable-X</code> branch
Line 178: Line 175:
# Deploy to gerrit1001 <code>scap deploy -l gerrit1001.wikimedia.org 'Gerrit to [version] on gerrit1001'</code>
# Deploy to gerrit1001 <code>scap deploy -l gerrit1001.wikimedia.org 'Gerrit to [version] on gerrit1001'</code>
# If you're only deploying plugins, you're done, otherwise SSH to the Gerrit master and issue <code>sudo service gerrit restart</code>
# If you're only deploying plugins, you're done, otherwise SSH to the Gerrit master and issue <code>sudo service gerrit restart</code>
XXX ensure plugins bundled in the war files get installed: <code>java -jar gerrit.war init --install-all-plugins</code>
XXX add a warning that init would cause upgrade steps to be conducted if need be (aka from a Gerrit version to another)

Revision as of 20:04, 10 May 2021

Since Feb 2nd 2021, we deploy the upstream war file as is. It is fetched from https://www.gerritcodereview.com/ . We still build extra plugins from sources though using the source code held in the Gerrit repository operations/software/gerrit.

The upstream branches (master, stable-3.2) are manually kept in sync. We have a forked branch wmf/stable-3.2 which has our extra plugins registered as sub-modules as well as some Wikimedia specific build scripts at the root of the repository.

Build prerequisites

Install OpenJDK 8, git, NodeJS, Python 3. On Debian:

$ sudo apt install git openjdk-8-jdk-headless nodejs python3

Gerrit and its plugins are build using Bazel. Since the exact version to use might differ between branches or can be updated by upstream at any point, one should install Bazelisk. It is a frontend to Bazel which automatically installs the needed Bazel version for the Gerrit branch you are working on. You will require the Go programming language in order to build it. The following will build Bazelisk 1.6.1 from source and copy the resulting binary in your user bin directory (~/.local/bin):

$ which go || sudo apt install golang-go
$ mkdir /tmp/bazelisk
$ (/tmp/bazelisk-build && go mod init . && go get github.com/bazelbuild/bazelisk@v1.6.1)
$ mkdir -p $HOME/.local/bin && cp /tmp/bazelisk-build/gopath/bin/bazelisk $HOME/.local/bin/bazelisk
$ rm -fR /tmp/bazelisk-build

Update our repository

Upgrade Gerrit core branch

  1. Clone Gerrit and submodules from Wikimedia add upstream as a remote
    git clone https://gerrit.wikimedia.org/r/operations/software/gerrit
cd gerrit
git remote add upstream https://gerrit.googlesource.com/gerrit
    Our repository HEAD points to the branch holding the material for the deployment deploy/wmf/stable-3.2
  2. Checkout the stable-3.2 branch (as of November 2020) which is a copy of the upstream release we currently use:
    git checkout stable-3.2 origin/stable-3.2
  3. Fetch the upstream branch and fast-forward our copy of their branch to the latest tag. Then push the stable-3.2 directly to Gerrit:
    git fetch --tags upstream stable-3.2
git merge --ff-only v3.2.3
git push --tags origin HEAD:stable-3.2
  4. Checkout wmf/stable-3.2 and merge the latest tag, push the resulting SINGLE merge commit up for review:
    git checkout wmf/stable-3.2
git merge v3.2.8

Update bundled submodules

Gerrit core comes with bundled plugins and a specific version of jgit which are registered as submodules. The .gitmodules file comes from upstream and uses relative urls. Since your work copy points to the Wikimedia Gerrit, the submodules would be fetched from our Gerrit which will fail since the repositories do not exist. Instead we have to instruct git to rewrite the URLs to use upstream repositories. This can be done using the git configuration setting url.<base>.insteadOf = <our url>.

  1. Ensure you are in our forked branch:
    git checkout wmf/stable-3.2
# update as needed
  2. Update git submodules from the upstream repository: 
    git -c url."https://gerrit.googlesource.com".insteadOf="$(dirname $(git config --get remote.origin.url))" submodule update --init
  3. Push for review. At this point, you will likely be ahead of wmf/stable-3.2 by...quite a bit (hundreds of patches). This is all within a single merge commit though, so Be Bold — push up your changes:
    git commit --ammend  # to add Change-Id and Phabricator tracking bug
git push origin HEAD:refs/for/wmf/stable-3.2

Fetch our additional plugins

As our Gerrit instance runs some plugins which are not part of Gerrit upstream set of plugins, we have those extra plugins registered and they need to be updated to match the new Gerrit core. You will need ssh admin access to Gerrit to be able to list the installed plugins as well as jq (sudo apt install jq).

List our plugins:

$ ./wmf-plugins-list.sh 
plugins/go-import
plugins/healthcheck
plugins/its-base
plugins/its-phabricator
plugins/javamelody
plugins/lfs
plugins/metrics-reporter-jmx
plugins/metrics-reporter-prometheus
plugins/motd
plugins/reviewers
plugins/zuul
$


Most plugins come with a set of stable- branches, some support multiple versions and solely have a master branch. The proper branch to use for our extra plugins is indicated in .gitmodules , we thus just have to update them from the remote branch (git submodule update --remote plugins/foo). To update our plugins:

$ ./wmf-plugins-update.sh
Updating Wikimedia specific plugins
<the list of our plugins is shown here>
<git submodule update if relevant>
Done
$

Fix-up of gitiles plugin

TODO we should fork it.

The gitiles plugin per default fetches fonts directly from Google. Since that's a third party dependency that we do not want, we instead want gitiles to use the fonts that Gerrit itself offers.

To achieve that, make sure your gitiles code includes change I4f4a0b7dd575cbc27641063e05d13b8a43a51d8b. The change did not get much traction upstream, so it probably won't land there. We nonetheless want to include that change.

Fix-up of javamelody plugin

TODO should not be needed

The javamelody plugin comes with external dependencies. Gerrit's build system does not automatically pick them up. To allow Gerrit to build the plugin, merge the contents of plugins/javamelody/external_plugin_deps.bzl into Gerrit's own plugins/external_plugin_deps.bzl


Once done updating our plugins submodules, git add them, update your commit and send it for review:

git add $(./wmf-plugins-list.sh)
git commit --amend
git push origin HEAD/refs/for/wmf/stable-3.2

Build Gerrit

Gerrit core

As of Feb 2nd 2021 we no more build Gerrit ourselves, we instead use upstream .war.

  1. Checkout the wmf/stable-3.2 (as of November 2020)
    git checkout -b wmf/stable-3.2 origin/wmf/stable-3.2

Production uses Java 8 as of November 19th 2020 which we will eventually switch to Java 11 ( https://phabricator.wikimedia.org/T268225 ). Meantime, Bazel has to be pointed to your local Java 8 installation: JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64.

  1. Build the bazel-bin/release.war
    JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 bazel build release

Note that if your building Gerrit v3.3 (or later) or master from 2020-10-06 (or later) and you're targeting Java 8, you'll also have to add the argument --java_toolchain=//tools:error_prone_warnings_toolchain (this is untested, but claimed on the gerrit mailing list).

Plugins

Most plugins come with build instructions usually available at src/main/resources/Documentation/build.html.

Since we use the upstream war, we do not need to build the plugins provided with Gerrit core. Instead we only build our extra plugins. The wmf-plugins-build.sh script iterate over the plugins found via wmf-plugins-list.sh and build them, taking care of dependencies files if needed:

$ ./wmf-plugins-list.sh 
plugins/go-import
plugins/healthcheck
plugins/its-base
plugins/its-phabricator
plugins/javamelody
plugins/lfs
plugins/metrics-reporter-jmx
plugins/metrics-reporter-prometheus
plugins/motd
plugins/reviewers
plugins/zuul
$

Build them:

$ ./wmf-plugins-build.sh
Building Wikimedia specific plugins

> Building go-import

...

Done building
Copying build artifacts
plugins-wmf/go-import.jar
plugins-wmf/healthcheck.jar
plugins-wmf/its-base.jar
plugins-wmf/its-phabricator.jar
plugins-wmf/javamelody.jar
plugins-wmf/lfs.jar
plugins-wmf/metrics-reporter-jmx.jar
plugins-wmf/metrics-reporter-prometheus.jar
plugins-wmf/motd.jar
plugins-wmf/reviewers.jar
plugins-wmf/zuul.jar

Those should be copied to the wmf/deploy branch

$

Upload artifacts

  1. Checkout the deploy/wmf/stable-X branch that aligns with the build branch
  2. Copy all the plugin jars from ./plugins-wmfover to ./plugins in the deploy branch
  3. Ensure Gerrit bundled plugins are not included and that any plugins we no more have installed are removed
  4. Commit the changes and upload to Gerrit
  5. Run ./upload_artifacts.py --version=3.2.8 gerrit.war plugins/foo.jar ... for the core application and all updated plugins (cf: Archiva, specifically setting up your ~/.m2/settings.xml
    Note: Alternatively, you can upload them by using the web UI, but that gets repetitive over a bunch of plugins hence why we wrote the above tool.

Deploying

  1. SSH to the deployment master (ssh deployment.eqiad.wmnet)
  2. Navigate to /srv/deployment/gerrit/gerrit
  3. Fetch & checkout the appropriate deploy/wmf/stable-X branch
  4. Deploy to gerrit2001 to ensure that there are no errors with git-fat et al scap deploy -l gerrit2001.wikimedia.org 'Gerrit to [version] on gerrit2001'
  5. Deploy to gerrit1001 scap deploy -l gerrit1001.wikimedia.org 'Gerrit to [version] on gerrit1001'
  6. If you're only deploying plugins, you're done, otherwise SSH to the Gerrit master and issue sudo service gerrit restart

XXX ensure plugins bundled in the war files get installed: java -jar gerrit.war init --install-all-plugins

XXX add a warning that init would cause upgrade steps to be conducted if need be (aka from a Gerrit version to another)

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=Gerrit/Upgrade&oldid=580401"