File:Anatomy of an mediawiki wiki dump airflow dag.jpg

Summary

Description	English: A mediawiki wiki dump DAG is composed of a first step, fetching the dump pod spec, as well as parallel dump pipelines, one per wiki, each running multiple stages. Each stage runs multiple dump jobs.
Date	30 April 2025
Source	Own work
Author	BRouberol-WMF

Licensing

I, the copyright holder of this work, hereby publish it under the following license:

This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.

You are free:

to share – to copy, distribute and transmit the work
to remix – to adapt the work

Under the following conditions:

attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
share alike – If you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original.

https://creativecommons.org/licenses/by-sa/4.0 CC BY-SA 4.0 Creative Commons Attribution-Share Alike 4.0 true true

This media file is uncategorized.

Please help improve this media file by adding it to one or more categories, so it may be associated with related media files ( how? ), and so that it can be more easily found.

Please notify the uploader with

{{subst:Please link images|File:Anatomy of an mediawiki wiki dump airflow dag.jpg}} ~~~~

Captions

English

Add a one-line explanation of what this file represents

Items portrayed in this file

depicts

creator<\/a>"}},"text\/plain":{"en":{"":"creator"}}},"{\"value\":{\"entity-type\":\"property\",\"numeric-id\":2093,\"id\":\"P2093\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"":" author name string<\/a>"}},"text\/plain":{"en":{"":"author name string"}}},"{\"value\":\"BRouberol-WMF\",\"type\":\"string\"}":{"text\/html":{"en":{"P2093":"BRouberol-WMF","P4174":" BRouberol-WMF<\/a>"}},"text\/plain":{"en":{"P2093":"BRouberol-WMF","P4174":"BRouberol-WMF"}}},"{\"value\":{\"entity-type\":\"property\",\"numeric-id\":4174,\"id\":\"P4174\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"":" Wikimedia username<\/a>"}},"text\/plain":{"en":{"":"Wikimedia username"}}},"{\"value\":{\"entity-type\":\"property\",\"numeric-id\":2699,\"id\":\"P2699\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"":" URL<\/a>"}},"text\/plain":{"en":{"":"URL"}}},"{\"value\":\"https:\\\/\\\/commons.wikimedia.org\\\/wiki\\\/User:BRouberol-WMF\",\"type\":\"string\"}":{"text\/html":{"en":{"P2699":" https:\/\/commons.wikimedia.org\/wiki\/User:BRouberol-WMF<\/a>"}},"text\/plain":{"en":{"P2699":"https:\/\/commons.wikimedia.org\/wiki\/User:BRouberol-WMF"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P170 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

some value

author name string : BRouberol-WMF

Wikimedia username : BRouberol-WMF

URL : https://commons.wikimedia.org/wiki/User:BRouberol-WMF

copyright status<\/a>"}},"text\/plain":{"en":{"":"copyright status"}}},"{\"value\":{\"entity-type\":\"item\",\"numeric-id\":50423863,\"id\":\"Q50423863\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"P6216":" copyrighted<\/a>"}},"text\/plain":{"en":{"P6216":"copyrighted"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P6216 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

copyrighted

copyright license<\/a>"}},"text\/plain":{"en":{"":"copyright license"}}},"{\"value\":{\"entity-type\":\"item\",\"numeric-id\":18199165,\"id\":\"Q18199165\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"P275":" Creative Commons Attribution-ShareAlike 4.0 International<\/a>"}},"text\/plain":{"en":{"P275":"Creative Commons Attribution-ShareAlike 4.0 International"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P275 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

Creative Commons Attribution-ShareAlike 4.0 International

source of file<\/a>"}},"text\/plain":{"en":{"":"source of file"}}},"{\"value\":{\"entity-type\":\"item\",\"numeric-id\":66458942,\"id\":\"Q66458942\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"P7482":" original creation by uploader<\/a>"}},"text\/plain":{"en":{"P7482":"original creation by uploader"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P7482 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

inception<\/a>"}},"text\/plain":{"en":{"":"inception"}}},"{\"value\":{\"time\":\"+2025-04-30T00:00:00Z\",\"timezone\":0,\"before\":0,\"after\":0,\"precision\":11,\"calendarmodel\":\"http:\\\/\\\/www.wikidata.org\\\/entity\\\/Q1985727\"},\"type\":\"time\"}":{"text\/html":{"en":{"P571":"30 April 2025"}},"text\/plain":{"en":{"P571":"30 April 2025"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P571 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

inception

30 April 2025

data size<\/a>"}},"text\/plain":{"en":{"":"data size"}}},"{\"value\":{\"amount\":\"+289486\",\"unit\":\"http:\\\/\\\/www.wikidata.org\\\/entity\\\/Q8799\"},\"type\":\"quantity\"}":{"text\/html":{"en":{"P3575":"289,486 byte<\/span>"}},"text\/plain":{"en":{"P3575":"289,486 byte"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P3575 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

data size

289,486 byte

height<\/a>"}},"text\/plain":{"en":{"":"height"}}},"{\"value\":{\"amount\":\"+1067\",\"unit\":\"http:\\\/\\\/www.wikidata.org\\\/entity\\\/Q355198\"},\"type\":\"quantity\"}":{"text\/html":{"en":{"P2048":"1,067 pixel<\/span>"}},"text\/plain":{"en":{"P2048":"1,067 pixel"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P2048 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

height

1,067 pixel

width<\/a>"}},"text\/plain":{"en":{"":"width"}}},"{\"value\":{\"amount\":\"+2048\",\"unit\":\"http:\\\/\\\/www.wikidata.org\\\/entity\\\/Q355198\"},\"type\":\"quantity\"}":{"text\/html":{"en":{"P2049":"2,048 pixel<\/span>"}},"text\/plain":{"en":{"P2049":"2,048 pixel"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P2049 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

width

2,048 pixel

media type<\/a>"}},"text\/plain":{"en":{"":"media type"}}},"{\"value\":\"image\\\/jpeg\",\"type\":\"string\"}":{"text\/html":{"en":{"P1163":"image\/jpeg"}},"text\/plain":{"en":{"P1163":"image\/jpeg"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P1163 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

media type

image/jpeg

checksum<\/a>"}},"text\/plain":{"en":{"":"checksum"}}},"{\"value\":\"2ac44871c2ed257ea88434c62c03fab3ba32e1a5\",\"type\":\"string\"}":{"text\/html":{"en":{"P4092":"2ac44871c2ed257ea88434c62c03fab3ba32e1a5"}},"text\/plain":{"en":{"P4092":"2ac44871c2ed257ea88434c62c03fab3ba32e1a5"}}},"{\"value\":{\"entity-type\":\"property\",\"numeric-id\":459,\"id\":\"P459\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"":" determination method or standard<\/a>"}},"text\/plain":{"en":{"":"determination method or standard"}}},"{\"value\":{\"entity-type\":\"item\",\"numeric-id\":13414952,\"id\":\"Q13414952\"},\"type\":\"wikibase-entityid\"}":{"text\/html":{"en":{"P459":" SHA-1<\/a>"}},"text\/plain":{"en":{"P459":"SHA-1"}}}}" class="wbmi-entityview-statementsGroup wbmi-entityview-statementsGroup-P4092 oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-framed">

checksum

2ac44871c2ed257ea88434c62c03fab3ba32e1a5

determination method or standard : SHA-1

Orientation	Normal
Horizontal resolution	144 dpi
Vertical resolution	144 dpi
Color space	sRGB