You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Gerrit: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Hashar
imported>Hashar
m ({{ptag|Gerrit}} ! :))
Line 1: Line 1:
{{gerrit nav}}
{{ptag|Gerrit}}
We use Gerrit for our code review system for Git.
We use Gerrit for our code review system for Git.


* [https://gerrit.wikimedia.org Gerrit]
* [https://gerrit.wikimedia.org Our Gerrit instance]
* [https://gerrit.wikimedia.org/r/Documentation/index.html Built-in Gerrit documentation]
* [https://gerrit.wikimedia.org/r/Documentation/index.html Built-in Gerrit documentation]
* [https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines Commit message guidelines]
* '''Monitoring'''
* '''Monitoring'''
** [https://gerrit.wikimedia.org/r/monitoring JavaMelody]
** [[gerrit:monitoring|JVM monitoring with JavaMelody]]
** [https://logstash.wikimedia.org/app/dashboards#/view/AW1f-0k0ZKA7RpirlnKV Kibana]
** [https://logstash.wikimedia.org/app/dashboards#/view/AW1f-0k0ZKA7RpirlnKV Server side logging (Kibana)]
** [https://grafana.wikimedia.org/dashboards/f/5AnaHr2Mk/gerrit Grafana]
** [https://grafana.wikimedia.org/dashboards/f/5AnaHr2Mk/gerrit Metrics dashboard (Grafana)]
** HTTP: [https://grafana.wikimedia.org/d/L0-l1o0Mz/apache?var-host=gerrit1001 Apache scoreboard] - [https://grafana.wikimedia.org/d/Zh_ncGsWk/queues-upstream?viewPanel=20 Gerrit HTTP threads view]
** HTTP: [https://grafana.wikimedia.org/d/L0-l1o0Mz/apache?var-host=gerrit1001 Apache scoreboard] - [https://grafana.wikimedia.org/d/Zh_ncGsWk/queues-upstream?viewPanel=20 Gerrit HTTP threads view]
 
Experimental:
 
*[[Gerrit test instance]]
* [[Gerrit test instance]]
* [https://gerrit.devtools.wmflabs.org/ Gerrit test install], hosted in [[Nova_Resource:Devtools|devtools Cloud VPS project]]
* [https://gerrit.devtools.wmflabs.org/ Gerrit test install], hosted in [[Nova_Resource:Devtools|devtools Cloud VPS project]]


== Details ==
== Overview ==
Gerrit is installed on [[gerrit1001]] in the prefix <tt>/var/lib/gerrit2</tt> and serves public request at https://gerrit.wikimedia.org/r/) . A replica runs at gerrit2001 which serves requests at https://gerrit-replica.wikimedia.org/r/ ), it is intended for a switch over and for various bots in order to offload the primary instance.
Gerrit is installed on [[gerrit1001]] in the prefix <code>/var/lib/gerrit2</code> and serves public request at https://gerrit.wikimedia.org/r/) . A replica runs on gerrit2001 which serves requests at https://gerrit-replica.wikimedia.org/r/ ), it is intended for a switch over and for various bots in order to offload the primary instance.


Gerrit is a Java daemon which listens for HTTP connections (port 8080) and SSH connections (port 29418).  Apache proxies the relevant URLs on port 80 and 443 through to Gerrit. The SSH port provides a restricted shell for Git checkouts and administrative commands.
Gerrit is a Java daemon which listens for HTTP connections (TCP port 8080) and SSH connections (TCP port 29418).  Apache proxies the relevant URLs on port 80 and 443 through to Gerrit. The SSH port provides a restricted shell for Git upload pack, reviews and administrative commands.


Since the Gerrit v3.2 upgrade in summer 2020, Gerrit no longer uses a conventional, relational database. So if you read somewhere about Gerrit's MySQL (or similar) database or "reviewdb", it is stale information. Instead of a relational database, Gerrit now stores the needed data directly in the git repositories (NoteDB) and to speed up lookup it creates indices to speed up access, also known has ''secondary index''. These indices are Lucene indices and H2 (flat file database engine from the Java world).
Since the Gerrit v3.2 upgrade in summer 2020, Gerrit no longer uses a conventional, relational database. So if you read somewhere about Gerrit's MySQL (or similar) database or "reviewdb", it is stale information. Instead of a relational database, Gerrit now stores the needed data directly in the git repositories (NoteDB). To speed up lookup, it creates indices also known has ''secondary index''. These indices are Lucene backed indices and H2 (flat file database engine from the Java world).


Git repositories are stored at <code>/srv/gerrit/git/</code>.
Git repositories are stored at <code>/srv/gerrit/git/</code>.
Line 32: Line 32:
As of November 2020, Gerrit runs with Java 8 (upgrade to Java 11 is {{Phabricator|T268225}}).
As of November 2020, Gerrit runs with Java 8 (upgrade to Java 11 is {{Phabricator|T268225}}).


== Using git ==
== End users documentations ==


See:
See:
Line 39: Line 39:
* Our own documentations:
* Our own documentations:
** [[mw:Gerrit/Tutorial]]
** [[mw:Gerrit/Tutorial]]
** [[mw:Gerrit/Commit_message_guidelines|Commit message guidelines]]
** [[mw:Gerrit/Advanced usage]]
** [[mw:Gerrit/Advanced usage]]


Line 47: Line 48:
See [[/Upgrade]].
See [[/Upgrade]].


== Restarting ==
== Administration ==
If you need to restart Gerrit (e.g. because it is OOMing), run <code>sudo systemctl restart gerrit</code>. This will take a few seconds before it comes back.
 
== Migrating ==
 
Example migration ticket (cobalt -> gerrit1001): https://phabricator.wikimedia.org/T222391
 
Topic branch from gerrit1001 migration: https://gerrit.wikimedia.org/r/q/topic:%22gerrit1001%22+(status:open%20OR%20status:merged)
 
Follow these steps to migrate from one server to another:
 
<big>'''Make sure you run rsync with --delete'''</big>
 
* Rsync /srv/gerrit/git/ , /srv/gerrit/plugins  and /var/lib/gerrit2/review_site/ from <old_host> to <new_host>
** <code>rsync --archive --verbose --delete /srv/gerrit/git/ rsync://<new_host>.wikimedia.org/gerrit-data/git/</code>
** <code>rsync --archive --verbose --delete /srv/gerrit/plugins/ rsync://<new_host>.wikimedia.org/gerrit-data/plugins/</code>
** <code>rsync --archive --verbose --delete /var/lib/gerrit2/review_site/ rsync://<new_host>.wikimedia.org/gerrit-var-lib/</code>
(Note: We are using rsync protocol directly and 'gerrit-data' and 'gerrit-var-'lib' are the names of rsync modules, not file system pathes. They are defined in "class profile::gerrit::migration".)
* Stop gerrit && disable puppet on <new_host>
* Create something similar to  https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/535966/
* Stop puppet on <old_host> + <replica>
* Create something similar to https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/541110/
* Create something similar to https://gerrit.wikimedia.org/r/#/c/operations/dns/+/541111/
* Stop gerrit on <old_host>
* Repeat the rsync commands above.
* Rename /var/lib/gerrit2/review_site/data/javamelody/r_<old_host> to /var/lib/gerrit2/review_site/data/javamelody/r_<new_host> on <new_host>.
* Run puppet on <new_host> + <old_host>
* Start gerrit on <new_host>
* [https://sal.toolforge.org/log/AW3wKZvNfYQT6VcDX2pS Hack DNS authdns-update] to clone from gerrit-replica temporarily, deploy DNS change
* Manually copy apache2 site config for gerrit.wm.org with scp from <old_host> to <new_host>, restart apache
* Manually run command from list_mediawiki_extensions cron to create /var/www/mediawiki-extensions.txt
* Run the online reindexer
 
* Decom <old_host> (create a ticket like https://phabricator.wikimedia.org/T236187)
 
== Creating new repositories ==
 
See [[mw:Git/Creating new repositories]]. Those without administrative access to Gerrit should use [[mw:Gerrit/New repositories]] instead.
 
== Inspecting an account settings ==
 
Since Gerrit 2.15, accounts are stored in <code>All-Users.git</code>. One first need to find the account reference, that can be done by checking which references a user can read on All-Users.git:
<syntaxhighlight lang="shell-session">
$ ssh -p 29418 gerrit.wikimedia.org gerrit ls-user-refs --project All-Users --user 'johndoe@wikimedia.org'
refs/meta/config
refs/users/34/1234
refs/users/self
$
</syntaxhighlight>
 
Note: for accounts having extended permissions on <code>All-Users.git</code>, that will show all references which is not useful.
 
Then on the server side (or remotely if you have privileges), you can list the configuration files and dump their content:
<syntaxhighlight lang="shell-session">
$ cd /srv/gerrit/git/All-Users.git
 
$ git ls-tree refs/users/34/1234
100644 blob xxx account.config
100644 blob yyy authorized_keys
100644 blob zzz preferences.confg
100644 blob ddd watch.config
 
$ git show refs/users/34/1234:authorized_keys
ssh-rsa AAAAxyz... RSA-1024
</syntaxhighlight>
 
The <code>All-Users</code> database can be browsed locally via git. The easiest way to find a user given the <code>All-Users</code> git repo is to git-grep for the username or email in the <code>refs/meta/external-ids</code> ref:
 
<syntaxhighlight lang="shell-session">
$ mkdir All-Users && cd All-Users && git init
$ git pull https://gerrit.wikimedia.org/r/All-Users refs/meta/external-ids
$ git grep johndoe
</syntaxhighlight>
 
== Become an Administrator ==
 
Gerrit management is granted by the [https://gerrit.wikimedia.org/r/#/admin/groups/1,members Administrators group] which includes the <code>gerrit2</code> system user and the LDAP group [http://tools.wmflabs.org/ldap/group/gerritadmin <code>gerritadmin</code>]. To gain Administrator right, one thus has to be added in LDAP.
 
Being an administrator grants all global capabilities as described at [https://gerrit.wikimedia.org/r/Documentation/access-control.html#global_capabilities Gerrit Access Control - global capabilities]. It is essentially the same as having root access on Gerrit.
 
The LDAP groups membership is cached in Gerrit for 1 hour (as of January 2020). One thus has to wait for the cache to expire or have an administrator to flush the cache:
 
    ssh -p 29418 gerrit.wikimedia.org gerrit flush-caches --cache ldap_groups
 
If right is given as a one off, one would most probably want to be removed from the group once the rights are no more required.
 
== Disabling / Blocking an account ==
 
=== Disable User ===
''This action is limited to Gerrit administrators.''
 
If an account needs to be disabled for some reason, there's two steps to complete:
* remove the ssh keys
* set the account to inactive
 
To set an account inactive, use this command:
$ ssh gerrit.wikimedia.org -p 29418 gerrit set-account --inactive <Username here>
 
To enable the user again:
 
$ ssh gerrit.wikimedia.org -p 29418 gerrit set-account --active <Username or user id here>
 
If you get an error that the user can't be found, try using the user id instead (can be obtained by grepping the all-users database).
 
And flush the accounts cache on the replica:
 
$ ssh gerrit-replica.wikimedia.org -p 29418 gerrit flush-caches --cache accounts
 
==== Flush all sessions ====
 
{{warn|'''Do not flush all sessions''' when banning a single user. As of May 2019 setting a user to inactive should invalidate their session automagically}}
 
Flushing <code>web_sessions</code> caches makes everyone re-login (aka, the sledgehammer). It should not be necessary in most circumstances; however, the command is still in this documentation in case it is needed in an unforeseen situation:
$ ssh -p 29418 gerrit.wikimedia.org gerrit flush-caches --cache web_sessions
 
== Firewalling Gerrit ==
On <code>gerrit1001.wikimedia.org</code> there is <code>/root/firewall.sh</code> and <code>/root/unfirewall.sh</code>, which will shut off access to Gerrit HTTP(S) and SSH.
Opsen will find them fairly self-explanatory.
 
== Tasks management ==
 
Gerrit queue tasks, given you have the appropriate permission you can run either of:
 
$ ssh -p 29418 gerrit.wikimedia.org ps -w
$ ssh -p 29418 gerrit.wikimedia.org gerrit show-queue -w
 
Tasks can be stuck which starve Gerrit processing. For example people would no more be able to fetch. In such a case, granted you are in the [https://gerrit.wikimedia.org/r/#/admin/groups/1,members Administrators group] you can kill a task by id:
 
$ ssh -p 29418 gerrit.wikimedia.org kill <SOMEID>
 
== Replication ==
 
=== Forcing Replication re-runs ===
 
When forcing a replication run through gerrit's replication plugin, stay clear of the <code>--all</code> switch, as it breaks renamed projects on github.
 
For example, both gerrit and github have a <code>mediawiki</code> repository. But github's <code>mediawiki</code> repository is actually <code>mediawiki/core</code> in gerrit. As gerrit also has a <code>mediawiki</code> repository, forcing replication on all projects would push gerrit's <code>mediawiki</code> repository onto github's <code>mediawiki</code> repository. Thereby, github's <code>mediawiki</code> repository falls over, and for examples the release tags from github's <code>mediawiki</code> are gone (cf. {{PhabT|100409}}).
 
This currently (2015-05-28) affects us for at least for the following repos:
{| class="wikitable"
|-
! Gerrit's name !! Overloaded name on GitHub
|-
| <code>mediawiki/core</code>
| <code>mediawiki</code>
|-
| <code>VisualEditor/VisualEditor</code>
| <code>VisualEditor</code>
|-
| <code>oojs/core</code>
| <code>oojs</code>
|}
 
If you by accident forced replication with <code>--all</code>, force replication of the effective gerrit repos to make up for it (i.e.: <code>mediawiki/core</code>, <code>VisualEditor/VisualEditor</code>, ...)
 
=== Github ===
 
As of July 2018 replication to github requires a repository to be manually created on github using the naming scheme gerrit expects, i.e. <code>/</code> replaced with <code>-</code> (FrEx: <code>operations/debs/trafficserver</code> -> <code>operations-debs-trafficserver</code>.
 
Github owners will be able to create these repositories.
 
Once a repository is created, replication to that repo can be forced using the gerrit replication ssh command.<syntaxhighlight lang=shell-session>
$ ssh -p 29418 gerrit.wikimedia.org replication start operations/debs/trafficserver --wait
</syntaxhighlight>
 
==== Troubleshooting ====
 
The group <code>mediawiki-replication</code> must be allowed to read <code>refs/tags/*</code> and <code>refs/heads/*</code> for replication to GitHub to work. This is because it is currently set as the <code>authGroup</code> in <code>replication.config</code> inside <code>etc</code> inside the Gerrit repository home.


Additionally, the <code>replication_log</code> should be checked in addition to the <code>gerrit.log</code> for mention of anything that could be wrong with replication.
See [[/Administration]]


Any changes to <code>replication.config</code> should be followed with a restart of the gerrit service to ensure that those changes are active. There is a bug with the replication plugin where it will reload the <code>replication.config</code> (and acknowledge that the configuration has been reloaded in the logs), but actually operate with a broken configuration in an unknown state until a restart.
== Operations ==


=== ORES ===
See [[/Operations]]
 
ORES mirroring setup is dangerous and unusual, but should, nevertheless, be documented.
 
The canonical repository for ORES lives on [https://github.com/wikimedia/ores github]. This is repository is observed by diffusion. This can done in the <code>Manage Repository</code> section of diffusion code browser. From there, phabricator "Mirrors" that repository (also available in diffusion management settings) to [https://gerrit.wikimedia.org/r/scoring/ores/ores gerrit].
 
This setup is discouraged.


== How to setup Gerrit in a Cloud VPS project ==
== How to setup Gerrit in a Cloud VPS project ==


See [[Gerrit/Running in Cloud VPS|Gerrit/Running in Cloud VPS]].
See [[Gerrit/Running in Cloud VPS|Gerrit/Running in Cloud VPS]].
== Monitoring ==
JavaMelody monitors the state of the Gerrit JVM. They are collected by Prometheus from https://gerrit.wikimedia.org/r/monitoring?prometheus
* [https://gerrit.wikimedia.org/r/monitoring JavaMelody in Gerrit] (only accesible to logged-in Gerrit Administrators/Gerrit Managers)
* [https://grafana.wikimedia.org/dashboards/f/5AnaHr2Mk Grafana "Gerrit" folder"]
** [https://grafana.wikimedia.org/d/Bw2mQ3iWz/gerrit-javamelody?orgId=1 Grafana Gerrit JavaMelody Dashboard]
** [https://grafana.wikimedia.org/d/uXZMn9PWz Overview]
=== Important Graphs ===
* Memory Usage
** [https://gerrit.wikimedia.org/r/monitoring?part=graph&graph=usedMemory Gerrit Used Memory]
** [https://grafana.wikimedia.org/d/Bw2mQ3iWz/gerrit-javamelody?panelId=12&fullscreen&orgId=1&var-Application=&var-Window=5m Grafana Used Memory]
* GC Timing
** [https://grafana.wikimedia.org/d/Bw2mQ3iWz/gerrit-javamelody?panelId=14&fullscreen&orgId=1  Grafana GC Timing]
** Garbage collection metrics. Times in the 100s of milliseconds, rather than in the 10s of milliseconds can be indicative of a problem (running low on memory)
* Active Threads
** [https://gerrit.wikimedia.org/r/monitoring?part=graph&graph=activeThreads Gerrit Active Threads]
** [https://grafana.wikimedia.org/d/Bw2mQ3iWz/gerrit-javamelody?panelId=16&fullscreen&orgId=1&var-Application=&var-Window=5m Grafana Active Threads]
** Usually there are less than 20 active threads at any given time — more than that typically means that you should take a [[#Thread_Dump|Thread Dump]] and restart.
=== Gerrit metrics ===
On top of the JavaMelody data, [https://gerrit.wikimedia.org/r/Documentation/metrics.html Gerrit has internal metrics].
For users having the [https://gerrit.wikimedia.org/r/Documentation/access-control.html#capability_viewCaches | viewCaches] or [https://gerrit.wikimedia.org/r/plugins/metrics-reporter-prometheus/Documentation/config.md View Metrics] capabilities, various internal Gerrit metrics can be retrieved via:
* https://gerrit.wikimedia.org/r/config/server/metrics
Which obviously requires authentication. That complements <code>gerrit show-caches</code>.
We use the metrics-reporter-prometheus plugin which exposes  collected by Prometheus from https://gerrit.wikimedia.org/r/plugins/metrics-reporter-prometheus/metrics . Those Gerrit metrics can also be seen on [https://gerrit.wikimedia.org/r/monitoring?part=mbeans the JavaMelody MBeans page] under the <code>metrics</code> branch.
See [https://grafana.wikimedia.org/dashboards/f/5AnaHr2Mk Gerrit Grafana dashboards folder].
=== Logs ===
Logs are available on the gerrit servers at: <code>/var/log/gerrit/</code>. There are a number of logfiles:
* <code>gerrit.log</code>: This is the main log file and will show stacktraces and errors
* <code>gerrit.json</code>: Like <code>gerrit.log</code> bug not really human readable. For sending structured logs to logstash.
* <code>sshd_log</code>: Log of sshd events
* <code>gc_log</code>: Logs for <code>git gc</code> '''not''' the JVM garbage collection (those logs are available in <code>/srv/gerrit/jvmlogs</code>)
* <code>plugin_log</code>: Info about plugins being loaded and reloaded, this information is also in <code>gerrit.log</code>
==== HTTP Logs ====
Gerrit sits behind apache. Apache access and error logs are both in <code>/var/log/apache2</code>:
* <code>gerrit.wikimedia.org.https.access.log</code>
* <code>gerrit.wikimedia.org.https.error.log</code>
==== Logstash ====
Gerrit's logs are sent to [[logstash]] using log4j. You can find its logs by searching with <code>type:log4j</code>.
=== Thread Dump ===
A thread dump is often useful in troubleshooting. To capture a thread dump use <code>jstack</code>. This code should be safe to run at any time, and is run frequently while Gerrit is running:
sudo -u gerrit2 jstack -l $(pgrep java) > "/srv/gerrit/jstack-$(date +%Y-%m-%d-%H%M%S).dump"
It's often useful to upload the resulting file to https://fastthread.io/ to detect problems.
=== Java trace ===
{{warn|This command isn't run very often, unsure how safe it is to run; kept here for folks who are familiar with jstat}}
Display a summary of garbage collection statistics every 1000 ms:
sudo -u gerrit2 /usr/lib/jvm/java-8-openjdk-amd64/bin/jstat -gcutil "$(pgrep -u gerrit2 java)" 1000
=== Java heap usage ===
''Requires openjdk-X-dbg for the debugging symbols''
  sudo /usr/lib/jvm/java-8-openjdk-amd64/bin/jmap -heap "$( pgrep -u gerrit2 java)"


== Troubleshooting / upgrade FAQ ==
== Troubleshooting / upgrade FAQ ==
Line 336: Line 86:
If you use [https://www.mediawiki.org/wiki/Gerrit/git-review git-review] to submit patches to gerrit, make sure you have 1.27 or newer (not 1.26).
If you use [https://www.mediawiki.org/wiki/Gerrit/git-review git-review] to submit patches to gerrit, make sure you have 1.27 or newer (not 1.26).
git-review 1.26 does not work with the new gerrit version.
git-review 1.26 does not work with the new gerrit version.
=== Access h2 account_patch_reviews ===
On copies of account_patch_reviews* files:
<pre>
java -cp h2-1.3.176.jar org.h2.tools.Shell -url jdbc:h2:/home/hashar/account_patch_reviews
</pre>
Which gives you a sql prompt:
<pre>
sql> show columns from ACCOUNT_PATCH_REVIEWS
...> ;
FIELD        | TYPE        | NULL | KEY | DEFAULT
ACCOUNT_ID  | INTEGER(10)  | NO  | PRI | 0
CHANGE_ID    | INTEGER(10)  | NO  | PRI | 0
PATCH_SET_ID | INTEGER(10)  | NO  | PRI | 0
FILE_NAME    | VARCHAR(255) | NO  | PRI | ''
(4 rows, 16 ms)
</pre>


== See also ==
== See also ==
* [[gerrit-wm]]
* [[Gerrit Notification Bot]]
* [[Gerrit Notification Bot]]
* [[Gerrit replica]]
* [[Gerrit replica]]


[[Category:Gerrit]]
[[Category:Gerrit]]

Revision as of 20:05, 10 May 2021

We use Gerrit for our code review system for Git.

Experimental:

Overview

Gerrit is installed on gerrit1001 in the prefix /var/lib/gerrit2 and serves public request at https://gerrit.wikimedia.org/r/) . A replica runs on gerrit2001 which serves requests at https://gerrit-replica.wikimedia.org/r/ ), it is intended for a switch over and for various bots in order to offload the primary instance.

Gerrit is a Java daemon which listens for HTTP connections (TCP port 8080) and SSH connections (TCP port 29418). Apache proxies the relevant URLs on port 80 and 443 through to Gerrit. The SSH port provides a restricted shell for Git upload pack, reviews and administrative commands.

Since the Gerrit v3.2 upgrade in summer 2020, Gerrit no longer uses a conventional, relational database. So if you read somewhere about Gerrit's MySQL (or similar) database or "reviewdb", it is stale information. Instead of a relational database, Gerrit now stores the needed data directly in the git repositories (NoteDB). To speed up lookup, it creates indices also known has secondary index. These indices are Lucene backed indices and H2 (flat file database engine from the Java world).

Git repositories are stored at /srv/gerrit/git/.

Gerrit uses the LDAP instance shared with Wikimedia Cloud Services for authentication. Accounts under ou=people,dc=wikimedia,dc=org and groups under ou=groups,dc=wikimedia,dc=org are exposed to Gerrit. The Gerrit account name is the cn field in LDAP. In order to log in to Gerrit, a user needs to already have a Wikimedia developer account. See Help:Getting Started for the process of getting a developer account.

To find out what version of Gerrit is running, you can use either of the following two commands:

java -jar /var/lib/gerrit2/review_site/bin/gerrit.war version
ssh -p 29418 gerrit.wikimedia.org gerrit version

As of November 2020, Gerrit runs with Java 8 (upgrade to Java 11 is task T268225).

End users documentations

See:

Alternatively, you can follow the Git & Gerrit codelab.

Upgrade procedures

See /Upgrade.

Administration

See /Administration

Operations

See /Operations

How to setup Gerrit in a Cloud VPS project

See Gerrit/Running in Cloud VPS.

Troubleshooting / upgrade FAQ

Frequently asked questions regarding issues since the recent Gerrit version upgrade to 3.2.

Plugin install error: TypeError: self.onAction is not a function

If you see this error please hard refresh your browser cache.

For some the Ctrl + F5 or Stift + Ctrl + F5 (or whatever combination your browser wants) is not sufficient.

If you still run into the issue after pressing your browsers F5 combination, please try clearing your caches for good.

For example on Firefox follow steps 1-6 from

If you use a different browser, it should allow you to clear cached web content somewhere in the setting too. Please find that in your browser's help pages and follow the instructions.

Can't upload any patches with git review

If you use git-review to submit patches to gerrit, make sure you have 1.27 or newer (not 1.26). git-review 1.26 does not work with the new gerrit version.

See also