You are browsing a read-only backup copy of Wikitech. The primary site can be found at

Nova Resource:Dumps/Documentation

From Wikitech-static
< Nova Resource:Dumps
Revision as of 00:11, 15 March 2019 by imported>ArielGlenn (→‎Project information)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search



This is a project that archives the public datasets generated by Wikimedia.


Archive the public Wikimedia datasets.

Anticipated time span


Project status

currently running

Contact address!forum/wikiteam-discuss

Willing to take contributors or not

not willing

Subject area narrow or broad


Project information


This project was created to provide a dedicated space just for transferring Wikimedia dump files to the Internet Archive. These dumps were created as a possible backup in the case of cluster-wide hardware failure, and its also often used by researchers/bots. Sometimes, these files are generated for forking of any Wikimedia project, when lots of people of a project has different aims from the original Wikimedia goal.

More information about the archiving process is available at Nova Resource:Dumps/

Data currently being archived

Here are some information and links regarding the data that this project is archiving:

  • Wikimedia main database dumps
  • Wikimedia incremental dumps
  • Wikidata JSON dumps
  • Wikimania videos
  • OpenStreetMap datasets


  • dumps-N (where N is an integer): Main archiving servers
  • dumps-stats: Wikimedia data manipulation, including dumps above and other stuff of relevance for Wikimedia research.


  • Before the eqiad migration we used to have a 900 GB quota (hardly sufficient for comfortable work).
  • Currently all heavy operations are conducted on /data/scratch/. We currently keep to a soft limit of using only 3 TB of space, but such disk usage is always temporary and will be deleted once the data is pushed to the Archive.
  • Everything is retained locally only for very short periods, just the time needed for packing on


Our resources:

Wikimedia data:

OSM data: