You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

WMDE/Wikidata/Dispatching

From Wikitech-static
< WMDE‎ | Wikidata
Revision as of 11:53, 4 November 2018 by imported>Addshore (start dispatching page)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Overview

  • Changes on Wikidata are buffered in the wb_changes table.
  • Dispatch state of wikis is stored in the wb_changes_dispatch table.
  • Multiple dispatchChanges.php scripts run creating ChangeNotificationJob jobs on wikis.
  • When the job runs the ChangeHandler handles the change, which includes cache purges, refreshing links and injecting rc records for example.

Full docs: https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/docs/change-propagation.wiki

The process runs on wikidatawiki and testwikidatawiki Does it run on beta?

Dispatch lag is linked to max lag. If dispatch lag increases max lag will increase and the edit rate on wikidata can dramatically drop. For example: https://grafana.wikimedia.org/dashboard/db/wikidata-edits?orgId=1&from=1541090549052&to=1541109255645

Control

The initiating of dispatchChanges.php is controlled by a cronjob in operations-puppet. This starts a new process every couple of minutes. Fine grained controll of the script can be done from within IS.php in mediawiki-config.

Stop dispatching

wmgWikibaseDispatchMaxTime can be set to 0, and after some minutes dispatching will stop.

If you really need to stop dispatching now change this setting and also kill the processes on the mwmaint server.

Monitoring

Dispatching state on the repo

Job queue for ChangeNotificationJobs

TODO dashboards for monitoring the job queue?