You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Portal:Data Services

From Wikitech-static
Revision as of 16:29, 20 May 2017 by imported>Madhuvishy (Add short summaries for each Data Service)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Data Services include services that allow for direct access to databases and dumps, and web interfaces for querying and programmatic access to data stores. The Data Services currently offered are Wiki Replicas (naming under discussion), ToolsDB, Wikimedia Dumps, Quarry and PAWS.

Wiki Replicas

Wiki Replicas are the sanitized public replicas of production MySQL Mediawiki databases. Access to the Wiki Replicas is granted for users with a ToolForge account automatically. See Help:Tool Labs/Database for how to access the Wiki Replicas.

ToolsDB

ToolsDB is a service that allows a Tool shared user to create and maintain a Tool specific database. See Help:Tool Labs/Database#User databases for help on ToolsDB.

Wikimedia Dumps

Wikimedia Dumps offers a range of data downloads including full text dumps, and other datasets. ToolForge users can directly access dumps data through their Tool account, see Help:Tool Labs#Dumps. VPS users can request to have the share available, see Help:Shared storage#.2Fpublic.2Fdumps

Quarry

Quarry is a graphical web interface that allows users to write SQL to query the Wiki Replicas. It only needs a Wikimedia (Meta) account to login, and is extensively used by analysts, researchers, and people of all experience levels to easily access the databases. See [[m:Research:Quarry]] for help.

PAWS

PAWS is a Juypter notebooks on the cloud service that hosts python notebooks and a terminal accessible through a web browser. It also only needs Meta account to login, and allows for access to the Wiki Replicas, ToolsDB and Dumps. See PAWS for help.

MediaWiki SQL database schema


https://wikitech.wikimedia.org/wiki/Help:Shared_storage

https://wikitech.wikimedia.org/wiki/Recover_instance_from_NFS

https://wikitech.wikimedia.org/wiki/Help:MySQL_queries

https://wikitech.wikimedia.org/wiki/Tool_Labs/Database_plan

https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database


Administration