You are browsing a read-only backup copy of Wikitech. The live site can be found at

shortcut: mwgrep

Wikimedia binaries: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
(→‎bin: Move scap information to scap wikitech page)
(→‎scap: stop mentioning sync-dir)
Line 13: Line 13:
[[Scap|<code>scap</code>]] is the Wikimedia deployment tool.
[[Scap|<code>scap</code>]] is the Wikimedia deployment tool.

'''subtasks''': {{anchor|sync-common}}[[Scap#scap_pull|<code>scap pull</code>]], {{anchor|sync-dir}}[[Scap#scap_sync-dir|<code>scap sync-dir</code>]], {{anchor|sync-file}}[[Scap#scap_sync-file|<code>scap sync-file</code>]], {{anchor|sync-wikiversions}}[[Scap#scap_sync-wikiversions|<code>scap sync-wikiversions</code>]], {{anchor|scap}}[[Scap#scap_sync|<code>scap sync</code>]]
'''subtasks''': {{anchor|sync-common}}[[Scap#scap_pull|<code>scap pull</code>]], {{anchor|sync-file}}[[Scap#scap_sync-file|<code>scap sync-file</code>]], {{anchor|sync-wikiversions}}[[Scap#scap_sync-wikiversions|<code>scap sync-wikiversions</code>]], {{anchor|scap}}[[Scap#scap_sync|<code>scap sync</code>]]

=== <code>mwgrep</code> ===
=== <code>mwgrep</code> ===

Revision as of 15:52, 7 September 2017

The following executable binaries are maintained in version control (operations/puppet:/files/misc/scripts) and copied/synced to servers (such as tin) by puppet into /usr/local/bin.



This script comes from puppet.


  • Runs a watch command on the hhvm logs with various exclusion and aggregation filters to monitor current php notices, warnings and errors on the cluster. This script lives on mwlog1001.


scap is the Wikimedia deployment tool.

subtasks: scap pull, scap sync-file, scap sync-wikiversions, scap sync


Search for CSS and JavaScript code fragments in MediaWiki, User and Module namespaces pages on all Wikimedia projects (powered by Elasticsearch).

It's located in operations/puppet/modules/scap/files/mwgrep


For convenience, this utility is also available as bin/expanddblist in any checkout of operations/mediawiki-config.git.

Usage: expanddblist [<dblist>, <dblistname> or <expression>]


./bin/expanddblist flow_computed > dblists/flow.dblist


An all-script is a script that operates on mediawiki-installation boxes via ssh. These scripts performs the action on all servers.

The specific servers include:


Sometimes you may need to purge a URL from Varnish cache.

In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time.

See also