WMDE/Wikidata/Caching

Varnish Cache

For full details of the varnish caching see Varnish &mw:Manual:Varnish_caching.

Purging a single page

echo 'https://www.wikidata.org/wiki/Special:EntityData/L1.rdf' | mwscript purgeList.php --wiki=aawiki

Purging a list of pages

mw:Manual:PurgeList.php

Purging all URLs in a file:

cat /home/addshore/listofURLs| mwscript purgeList.php --wiki=aawiki

Purging all pages in a (the Lexeme) namespace:

mwscript purgeList.php --wiki wikidatawiki --namespace 146

Parser Cache

mw:Manual:$wgCacheEpoch is used by many other things than just the parser cache. So DON'T TOUCH it....

Purging a selection of numeric Ids

for i in {1..1000}; do echo Lexeme:L$i; done | mwscript purgePage.php --wiki wikidatawiki --skip-exists-check

Selectively reject mass parser cache values

If you want to selectively reject parser cache entries you can use the RejectParserCacheValue hook.

This can be added to CommonSettings.php, for example for example https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/468301/1/wmf-config/CommonSettings.php

// T203888: Purge Wikidata Lexeme parser cache for senses deployment - Addshore
if ( $wgDBname === 'wikidatawiki' ) {
	/** @var WikiPage $wikiPage */
	$wgHooks['RejectParserCacheValue'][] = function ( $value, $wikiPage, $popts ) {
		if (
			$wikiPage->getTitle()->getNamespace() === 146 &&
			$value->expired( '20181018105500' )
		) {
			return false;
		}
		return true;
	};
}

You can monitor the rejection rate for the whole cluster on https://grafana.wikimedia.org/dashboard/db/parser-cache

If you are going to be rejecting a large amount of entries / a large period of time then ideally ping the DBAs before doing so.

Actively remove parser cache entries

You can use mw:Manual:PurgeParserCache.php.

On the WMF cluster this already runs nightly.

Shared entity cache

The shared entity cache shares

From Wikibase/docs/options.wiki, docs for $wgWBRepoSettings and $wgWBClientSettings keys.

;sharedCacheKeyPrefix:  Prefix to use for cache keys that should be shared among a wikibase repo and all its clients. The default is constructed from <code>$wgDBname</code> and <code>WBL_VERSION</code>. In order to share caches between clients (and the repo), set a prefix based on the repo's name and <code>WBL_VERSION</code> or a similar version ID.
:'''Note''': The default may change in order to use the repo's database name automatically.
;sharedCacheDuration: The default duration of entries in the shared object cache, in seconds. Default is 3600 seconds (1 hour).
;sharedCacheType: The type of cache to use for the shared object cache. Defaults to <code>$wgMainCacheType</code>. Use <code>CACHE_XXX</code> constants.

The shared cache key in production is set in wmf-config/Wikibase.php and will look something like "wikibase_shared/1_31_0-wmf_2-wikidatawiki-hhvm"

This key is used as a prefix for various other entity related caches that could benefit from being cached across the whole cluster.