You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Infographics: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Quiddity
imported>Krinkle
Line 1: Line 1:
:''See also: [[Special:ListFiles]]''
:''See also: [[Special:ListFiles]]''
:''See also: [[c:Category:Wikimedia technology]] -> [[c:Category:Wikimedia technical operations]] -> [[c:Category:Wikimedia Cloud Services]] (etc)''
:''See also: [[c:Category:Wikimedia technology|commons:Category:Wikimedia technology]] -> [[c:Category:Wikimedia technical operations|commons:Category:Wikimedia technical operations]] -> [[c:Category:Wikimedia Cloud Services|commons:Category:Wikimedia Cloud Services]] (etc)''
:''See also: [[Performance/Runbook/diagrams.net conventions]]''
:''See also: [[Performance/Runbook/diagrams.net conventions]]''


This page is a compilation of images from scattered locations. Currently in Chronological order. Update/overhaul freely!
== Data center overview ==
{{Main|Data centers}}
<gallery>
File:Infrastructure overview 18 January 2016 (draft).png|alt=|2016-01-18
File:Wikimedia-servers-2010-12-28.svg|2010-12-28
File:Wikimedia-servers-2009-04-05.svg|2009-04-05
File:Wikimedia-servers-2008-11-10.svg|2008-11-10
File:Wikimedia Servers.svg|2006-03-12
File:Wikimedia-servers-2006-05-09.svg|2006-05-09
File:Wikimedia cluster.jpg|alt=|2005-11
File:Wikimedia-servers-2005-04-12.png|2005-04-12
File:Wikimedia-servers-2005-01-30.png|2005-01-30
File:Wikimedia-servers-2004-12-10.png|2004-12-10
File:Wikimedia-servers-2004-11-11.png|2004-11-11
File:Wikimedia-servers-2004-08-26.png|2004-08-26
File:Wikimedia-servers-2004-08-24.png|2004-08-24
File:Wikimedia-servers-2004-02-19-800wide.png|2004-02-19
</gallery>
 
== MediaWiki architecture ==
{{Main|MediaWiki at WMF}}<gallery>
File:Wikipedia webrequest flow 2020.png|April 2020
File:Wikipedia webrequest flow 2015-10.png|October 2015
File:Wikimedia Server Architecture (simplified).svg|October 2013: core architecture of a single datacenter.
</gallery>


== Traffic ==
{{Main|Global traffic routing}}<gallery>
File:WMF Varnish and Swift.svg|alt=|2015-12-04 Varnish and Swift
File:WMF Inbound Text Traffic Diagram.svg|2015-02-02
</gallery>
== Networking ==
{{Main|Network design}}<gallery>
File:Wikimedia network overview.png|alt=|2022-02 (includes back to 2018 in previous file revisions).
File:Wikimedia Ashburn DC, physical layer.png|alt=|2018-05: Ashburn DC physical layer.
File:AS43821 2010.png|alt=|2010-02: Amsterdam network
File:SFO office network Jan 2009.png|alt=|2009-01: SF office network.
</gallery>
== Individual software, services, and clusters ==
<gallery>
<gallery>
File:New service IP flow chart.png|New service IP flow chart - 2021-04
File:New service IP flow chart.png|New service IP flow chart - 2021-04
File:Wikimedia network overview.png|Wikimedia network overview - 2021-03 (more at [[Network design]])
File:Wikifunctions - Top-level architectural model.svg|Wikifunctions - Top-level architectural model - 2021-01
File:Wikifunctions - Top-level architectural model.svg|Wikifunctions - Top-level architectural model - 2021-01
File:WMCS network-L2 L3.png|WMCS network-L2 L3 - 2020-10
File:WMCS network-L2 L3.png|WMCS network-L2 L3 - 2020-10
File:MediaWiki fallback chains.svg|MediaWiki language fallback chains - 2020-07
File:MediaWiki fallback chains.svg|MediaWiki language fallback chains - 2020-07
File:PAWS Design.png|basic setup and flows of the newer PAWS cluster - 2020-07
File:PAWS Design.png|basic setup and flows of the newer PAWS cluster - 2020-07
File:Wikipedia_webrequest_flow_2020.png|Wikipedia webrequest flow - 2020-04
File:ResourceLoader Client lifecycle 2020.png|ResourceLoader Client lifecycle - 2020-04
File:ResourceLoader_Client_lifecycle_2020.png|ResourceLoader Client lifecycle - 2020-04
File:Modern Event Platform Architecture Diagram.jpeg|Event Platform Architecture - 2020-04
File:Analytics Kerberos infra scheme.svg|Analytics Kerberos infrastructure - 2019-11
File:Analytics Kerberos infra scheme.svg|Analytics Kerberos infrastructure - 2019-11
File:Modern Event Platform Architecture Diagram.jpeg|Event Platform Architecture - 2019-11
File:Infrastructure overview.png|Infrastructure overview - 2019-10
File:Toolforge k8s network topology.png|Toolforge k8s network topology - 2019-10
File:Toolforge k8s network topology.png|Toolforge k8s network topology - 2019-10
File:RBACandPSPforToolforge.png|Pod Security Policies (proposal) - 2019-09
File:RBACandPSPforToolforge.png|Pod Security Policies (proposal) - 2019-09
File:Database_backups_overview.svg|Architecture of Wikimedia Foundation infrastructure Database backups - 2019-03
File:Database backups overview.svg|Architecture of Wikimedia Foundation infrastructure Database backups - 2019-03
File:CirrusSearch components.svg|CirrusSearch components diagram - 2019-02
File:CirrusSearch components.svg|CirrusSearch components diagram - 2019-02
File:Wikidata Query Service components.svg|Component diagram for Wikidata Query Service - 2019-02
File:Wikidata Query Service components.svg|Component diagram for Wikidata Query Service - 2019-02
Line 30: Line 66:
File:Eqiad1 network topology.png|network diagram for the CloudVPS eqiad1 openstack deployment - 2018-08
File:Eqiad1 network topology.png|network diagram for the CloudVPS eqiad1 openstack deployment - 2018-08
File:OpenStack at WMCS.pdf|OpenStack at WMCS - 2018-05
File:OpenStack at WMCS.pdf|OpenStack at WMCS - 2018-05
File:Wikimedia Ashburn DC, physical layer.png|Wikimedia Ashburn DC, physical layer - 2018-05 (more at [[Network design]])
File:WDCM System Operation Workflow.png|WDCM System Operation Workflow: an overview of the WDCM monthly update - 2018-01
File:WDCM_System_Operation_Workflow.png|WDCM System Operation Workflow: an overview of the WDCM monthly update - 2018-01
File:Wm technical areas.svg|Wikimedia Technical Areas - 2018-05
File:Wm_technical_areas.svg|Wikimedia Technical Areas - 2018-05
File:Logic model for onboarding new developers annual program.svg|Logic model for onboarding new developers annual program - 2017-6
File:Logic model for onboarding new developers annual program.svg|Logic model for onboarding new developers annual program - 2017-6
File:Containerized continuous delivery 2017 concept.png|Containerized continuous delivery 2017 concept - 2017
File:Containerized continuous delivery 2017 concept.png|Containerized continuous delivery 2017 concept - 2017
File:Wikireplicas.png|Architecture of wikireplicas - 2017-11
File:Wikireplicas.png|Architecture of wikireplicas - 2017-11
File:Frontend resource loading in MediaWiki.png|Simple overview image about how MW loads resources in clients - 2017-11
File:Frontend resource loading in MediaWiki.png|Simple overview of how MW loads cliient-side resources, 2017-11.
File:New_article_routing.with_ORES.svg|maps the flow of new articles in Wikipedia with the 'draftquality' and 'articletopic' ORES models used for routing - 2017-08
File:New article routing.with ORES.svg|maps the flow of new articles in Wikipedia with the 'draftquality' and 'articletopic' ORES models used for routing - 2017-08
File:Labs Tools topology.png|labs tools topology - 2016-10
File:Labs Tools topology.png|labs tools topology - 2016-10
File:Maps-components.png|Maps-components - 2016-08
File:Maps-components.png|Maps-components - 2016-08
Line 44: Line 79:
File:Irc-rcstream-sequence.svg|Irc-rcstream-sequence (draft) - 2016-02
File:Irc-rcstream-sequence.svg|Irc-rcstream-sequence (draft) - 2016-02
File:WMF Varnish and Swift.svg|Diagram of traffic flow for image traffic through Varnish to Swift - 2015-12
File:WMF Varnish and Swift.svg|Diagram of traffic flow for image traffic through Varnish to Swift - 2015-12
File:Wikipedia webrequest flow 2015-10.png|Wikipedia webrequest flow 2015-10
File:Scap-diagram.png|Scap-diagram - 2015-04
File:Scap-diagram.png|Scap-diagram - 2015-04
File:MediaWiki 1.24.1 database schema.svg|database schema - 2015-03 - See dynamically updated version at [[mw:Manual:Database layout/diagram]] (Old: [[dbtree.wikimedia.org]])
File:MediaWiki 1.24.1 database schema.svg|2015-03: MediaWiki database schema. Latest version at [[mw:Manual:Database layout/diagram|mw:Manual:Database layout]].
File:WMF Inbound Text Traffic Diagram.svg|WMF Inbound Text Traffic Diagram - 2015-02
File:Analytics Cluster Diagram.png|Analytics Cluster Webrequest ingestion diagram - 2014-07
File:Analytics Cluster Diagram.png|Analytics Cluster Webrequest ingestion diagram - 2014-07
File:Wikimedia Server Architecture (simplified).svg|simplified version of the Wikimedia server infrastructure, showing the core architecture of a single datacenter - 2013-10
File:Logging current.png|Logging current - 2013-08
File:Logging current.png|Logging current - 2013-08
File:Logging future.png|Logging future - 2013-08
File:Logging future.png|Logging future - 2013-08
Line 59: Line 91:
File:Thumbnail request path all swift v3.jpg|Thumbnail request path all swift v3 - 2011-08
File:Thumbnail request path all swift v3.jpg|Thumbnail request path all swift v3 - 2011-08
File:Virtual architecture diagram.svg|Test/Dev Architecture - 2011-04 (See also [[:File:The Site Architecture You Can Edit.pdf]])
File:Virtual architecture diagram.svg|Test/Dev Architecture - 2011-04 (See also [[:File:The Site Architecture You Can Edit.pdf]])
File:Wikimedia-servers-2010-12-28.svg|Wikimedia servers - 2010-12
File:AS43821 2010.png|Wikimedia Amsterdam Network - 2010-02
File:SFO office network Jan 2009.png|SFO office network - 2009-01
File:Wikimedia cluster.jpg|Wikimedia cluster - 2005-11
</gallery>
</gallery>


{{ClusterMap|thumb=1}}
== See also ==


== See also ==
* [[File:Commons-logo.svg|border|20x20px]][[c:Category:Wikimedia_servers_by_location|Wikimedia data center photos]] and [[c:Category:Wikimedia servers diagrams|Wikimedia server diagrams]] on Wikimedia Commons.


Also:
* [[:Category:Wikimedia infrastructure]]
* call graphs [https://lists.wikimedia.org/pipermail/wikitech-l/2016-November/086971.html by Seb35] in 2016
* Call graphs [[mailarchive:wikitech-l/2016-November/086971.html|by Seb35]] in 2016
** [[mw:File:Call graph of extension Math.svg]]
** [[mw:File:Call graph of extension Math.svg]]
** [[mw:File:Call graph of extension CentralAuth.svg]]
** [[mw:File:Call graph of extension CentralAuth.svg]]
** [[mw:File:Call graph of extension MediaWikiFarm.svg]]
** [[mw:File:Call graph of extension MediaWikiFarm.svg]]
More graphics:
* [[:Category:Wikimedia infrastructure]]
* [[c:Category:Wikimedia servers diagrams]]
* [[m:Server layout diagrams]]

Revision as of 18:34, 20 June 2022

See also: Special:ListFiles
See also: commons:Category:Wikimedia technology -> commons:Category:Wikimedia technical operations -> commons:Category:Wikimedia Cloud Services (etc)
See also: Performance/Runbook/diagrams.net conventions

Data center overview

Main article: Data centers

  • 2016-01-18

  • 2010-12-28

    2010-12-28

  • 2009-04-05

    2009-04-05

  • 2008-11-10

    2008-11-10

  • 2006-03-12

    2006-03-12

  • 2006-05-09

    2006-05-09

  • 2005-11

  • 2005-04-12

    2005-04-12

  • 2005-01-30

    2005-01-30

  • 2004-12-10

    2004-12-10

  • 2004-11-11

    2004-11-11

  • 2004-08-26

    2004-08-26

  • 2004-08-24

    2004-08-24

  • 2004-02-19

    2004-02-19

MediaWiki architecture

Main article: MediaWiki at WMF
  • April 2020

    April 2020

  • October 2015

    October 2015

  • October 2013: core architecture of a single datacenter.

    October 2013: core architecture of a single datacenter.

    • Traffic

    Main article: Global traffic routing

    • 2015-12-04 Varnish and Swift

    • 2015-02-02

      2015-02-02

      • Networking

      Main article: Network design

      • 2022-02 (includes back to 2018 in previous file revisions).

      • 2018-05: Ashburn DC physical layer.

      • 2010-02: Amsterdam network

      • 2009-01: SF office network.

        • Individual software, services, and clusters

        • New service IP flow chart.png

          New service IP flow chart - 2021-04

        • Wikifunctions - Top-level architectural model - 2021-01

          Wikifunctions - Top-level architectural model - 2021-01

        • WMCS network-L2 L3 - 2020-10

          WMCS network-L2 L3 - 2020-10

        • MediaWiki language fallback chains - 2020-07

          MediaWiki language fallback chains - 2020-07

        • basic setup and flows of the newer PAWS cluster - 2020-07

          basic setup and flows of the newer PAWS cluster - 2020-07

        • ResourceLoader Client lifecycle - 2020-04

          ResourceLoader Client lifecycle - 2020-04

        • Event Platform Architecture - 2020-04

          Event Platform Architecture - 2020-04

        • Analytics Kerberos infrastructure - 2019-11

          Analytics Kerberos infrastructure - 2019-11

        • Toolforge k8s network topology - 2019-10

          Toolforge k8s network topology - 2019-10

        • Pod Security Policies (proposal) - 2019-09

          Pod Security Policies (proposal) - 2019-09

        • Architecture of Wikimedia Foundation infrastructure Database backups - 2019-03

          Architecture of Wikimedia Foundation infrastructure Database backups - 2019-03

        • CirrusSearch components diagram - 2019-02

          CirrusSearch components diagram - 2019-02

        • Component diagram for Wikidata Query Service - 2019-02

          Component diagram for Wikidata Query Service - 2019-02

        • Wikidata: High Level - 2018-08 (see blog post for more details)

          Wikidata: High Level - 2018-08 (see blog post for more details)

        • Wikidata: Data getting into Wikidata - 2018-08

          Wikidata: Data getting into Wikidata - 2018-08

        • Wikidata: Data getting to client sites - 2018-08

          Wikidata: Data getting to client sites - 2018-08

        • Wikidata: Dumps - 2018-08

          Wikidata: Dumps - 2018-08

        • Wikidata: Query Service - 2018-08

          Wikidata: Query Service - 2018-08

        • example of NICs as arranged for Neutron in the labtestn Openstack deployment - 2018-08

          example of NICs as arranged for Neutron in the labtestn Openstack deployment - 2018-08

        • network diagram for the CloudVPS eqiad1 openstack deployment - 2018-08

          network diagram for the CloudVPS eqiad1 openstack deployment - 2018-08

        • OpenStack at WMCS - 2018-05

          OpenStack at WMCS - 2018-05

        • WDCM System Operation Workflow: an overview of the WDCM monthly update - 2018-01

          WDCM System Operation Workflow: an overview of the WDCM monthly update - 2018-01

        • Wikimedia Technical Areas - 2018-05

          Wikimedia Technical Areas - 2018-05

        • Logic model for onboarding new developers annual program - 2017-6

          Logic model for onboarding new developers annual program - 2017-6

        • Containerized continuous delivery 2017 concept - 2017

          Containerized continuous delivery 2017 concept - 2017

        • Architecture of wikireplicas - 2017-11

          Architecture of wikireplicas - 2017-11

        • Simple overview of how MW loads cliient-side resources, 2017-11.

          Simple overview of how MW loads cliient-side resources, 2017-11.

        • maps the flow of new articles in Wikipedia with the 'draftquality' and 'articletopic' ORES models used for routing - 2017-08

          maps the flow of new articles in Wikipedia with the 'draftquality' and 'articletopic' ORES models used for routing - 2017-08

        • labs tools topology - 2016-10

          labs tools topology - 2016-10

        • Maps-components - 2016-08

          Maps-components - 2016-08

        • Maps-deployment - 2016-08

          Maps-deployment - 2016-08

        • Irc-rcstream-deployment (draft) - 2016-02

          Irc-rcstream-deployment (draft) - 2016-02

        • Irc-rcstream-sequence (draft) - 2016-02

          Irc-rcstream-sequence (draft) - 2016-02

        • Diagram of traffic flow for image traffic through Varnish to Swift - 2015-12

          Diagram of traffic flow for image traffic through Varnish to Swift - 2015-12

        • Scap-diagram - 2015-04

          Scap-diagram - 2015-04

        • 2015-03: MediaWiki database schema. Latest version at mw:Manual:Database layout.

          2015-03: MediaWiki database schema. Latest version at mw:Manual:Database layout.

        • Analytics Cluster Webrequest ingestion diagram - 2014-07

          Analytics Cluster Webrequest ingestion diagram - 2014-07

        • Logging current - 2013-08

          Logging current - 2013-08

        • Logging future - 2013-08

          Logging future - 2013-08

        • World-wide - 2013-08

          World-wide - 2013-08

        • Labs Projects Instance - 2013-08

          Labs Projects Instance - 2013-08

        • The briefer but less terrifying life of a puppet patch - 2013-07

          The briefer but less terrifying life of a puppet patch - 2013-07

        • External storage cluster - 2011-11

          External storage cluster - 2011-11

        • External storage cluster - 2011-11

          External storage cluster - 2011-11

        • Thumbnail request path all swift v3 - 2011-08

          Thumbnail request path all swift v3 - 2011-08

        • Test/Dev Architecture - 2011-04 (See also File:The Site Architecture You Can Edit.pdf)

          Test/Dev Architecture - 2011-04 (See also File:The Site Architecture You Can Edit.pdf)

        See also

        Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=Infographics&oldid=591504"