You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Wikimedia binaries: Difference between revisions
imported>Gergő Tisza (→mwgrep) |
imported>Krinkle (Document sql command) |
||
| Line 69: | Line 69: | ||
Advanced usage example using <code>expanddblist</code> to parallelise <code>foreachwiki</code>: | Advanced usage example using <code>expanddblist</code> to parallelise <code>foreachwiki</code>: | ||
/usr/local/bin/expanddblist all | xargs -I{} -P 4 sh -c \'/usr/local/bin/mwscript extensions/Example/maintenance/updateThing.php --wiki={} > /var/log/mediawiki/example-thing/{}.log 2>&1 | /usr/local/bin/expanddblist all | xargs -I{} -P 4 sh -c \'/usr/local/bin/mwscript extensions/Example/maintenance/updateThing.php --wiki={} > /var/log/mediawiki/example-thing/{}.log 2>&1 | ||
=== <code>sql</code> === | |||
{{shortcut|sql}} | |||
See [[Debugging in production#Debugging databases]]. | |||
== Purge == | == Purge == | ||
Revision as of 19:05, 22 December 2021
 | This page may be outdated or contain incorrect details. Please update it if you can. |
The following executable binaries are maintained in version control (operations/puppet:/modules/scap/files and others) and provisioned by Puppet into /usr/local/bin.
Deployment host
These commands on the deployment host (deployment.eqiad.wmnet).
httpbb
View source in gerrit.
mwlog-server
These commands must be run from mwlog1001.
fatalmonitor
No longer exists - see logspam-watch instead.
logspam-watch
Runs a watch command over logspam, which in turn filters and aggregates various PHP error and exception logs.
Maintenance host
These commands must be run on a maintenance host (e.g. mwmaint1002).
Consider using screen before starting any long-running maintenance scripts.
mwscript
Maintenance script runner. This wrapper is mandatory for all maintenance scripts in production. It takes care of finding where the appropriate MediaWiki installation on the server for the selected dbname (per Multiversion), setting environment variables, sudo-ing as the correct shell user, and more. See Heterogeneous deployment for more about Multiversion.
Script names must be relative to a MediaWiki install path, for example:
mwscript maintenance/doExample.php --wiki <nameofwiki> [args]
or
mwscript extensions/Example/doExample.php --wiki <nameofwiki> [args]
mwscriptwikiset
Sequentially run a maintenance script for all wikis in a dblist-file (in wmf-config/dblists). For example:
mwscriptwikiset maintenance/doExample.php group0.dblist [args]
This only supports expanded or plain dblist files. It does not support dblist files containing a computable expression.
It effectively runs mwscript maintenance/doExample.php <nameofwiki> [args].
foreachwikiindblist
Sequentially run a maintenance script for all wikis in a dblist expression.
Similar to mwscriptwikiset, except that it supports dblist expressions, and invokes mwscript with --wiki instead of position arguments.
For example:
foreachwikiindblist <dblist or expression> maintenance/doExample.php [args]
Will effectively run mwscript --wiki <nameofwiki> maintenance/doExample.php [args].
foreachwiki
Sequentially run a maintenance script for all wikis in the realm. If run from mwmaint1002, this will use all.dblist. If run from Beta Cluster, it uses all-labs.dblist.
This is effectively a shortcut for foreachwikiindblist <all or all-labs> [args] .
For example:
foreachwiki maintenance/doExample.php [args]
mwgrep
Search for a pattern in the revision text of all Wikimedia projects, specifically for code-pages with JS or CSS format in the "MediaWiki", "User" or "Module" namespace. (Powered by Elasticsearch).
It's located in operations/puppet/modules/scap/files/mwgrep
See also global-search.toolforge.org, a public tool with more-or-less equivalent functionality.
expanddblist
For convenience, this utility is also available locally as bin/expanddblist in any checkout of operations/mediawiki-config.git.
Usage: expanddblist [<dblist>, <dblistname> or <expression>]
Example:
./bin/expanddblist flow_computed > dblists/flow.dblist
Advanced usage example using expanddblist to parallelise foreachwiki:
/usr/local/bin/expanddblist all | xargs -I{} -P 4 sh -c \'/usr/local/bin/mwscript extensions/Example/maintenance/updateThing.php --wiki={} > /var/log/mediawiki/example-thing/{}.log 2>&1
sql
See Debugging in production#Debugging databases.
Purge
Sometimes you may need to purge a URL from Varnish cache.
In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time.