You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

LVS and Varnish: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Ema
imported>Krinkle
No edit summary
Line 4: Line 4:
[[File:WMF Inbound Text Traffic Diagram.svg|750px|Diagram of "text" traffic flow through Wikimedia front edge LVS/nginx/Varnish infrastructure.]]
[[File:WMF Inbound Text Traffic Diagram.svg|750px|Diagram of "text" traffic flow through Wikimedia front edge LVS/nginx/Varnish infrastructure.]]


=== Notes ===
=== Naming ===
[[esams]] is representative of all cache-only sites; [[ulsfo]] is similar.


"[[upload.wikimedia.org|upload]]" and "misc" traffic operate similarly to "text" shown here.
* [[Eqiad cluster|eqiad]] is representative of whichever data centre is currently primary ([[codfw]] is similar).
* [[esams]] is representative of all caching sites ([[ulsfo]] is similar).


When [[LVS]] balances traffic to ports :80 ([[varnish]]es), and :443 (nginxes), it uses a hash of the client IP to help with TCP Fast Open and SSL session persistence respectively.
* "[[upload.wikimedia.org|upload]]" and "misc" traffic operate similarly to "text" shown here.


Within the caching layer (cp[1234]xxx machines), the jump from [[nginx]] to varnish:80 is direct on the local host only.
See [[Varnish]] and [[Clusters]] for more information about those in particular.


However, the jump from varnish:80 (frontend) to varnish:3128 (backend) is different: for that jump, we hash on the URL (etc) when balancing to the backends to divide the cache space among all machines, and thus the request typically moves from one machine to another within the cache layer here.
=== Routing ===
When [[LVS]] balances traffic to ports :80 (varnish), and :443 (nginx), it uses a hash of the client IP to help with TCP Fast Open and SSL session persistence respectively.
 
Within the caching layer (cp#xxx machines), the jump from [[nginx]]:443 to varnish:80 is direct on the local host.
 
However, the jump from varnish:80 (frontend) to varnish:3128 (backend) is different: for that jump, we hash on the URL (and other req meta-data) when balancing to the backends to divide the cache space among all machines, and thus the request typically moves from one machine to another within the same cluster.


[[Category:Caching]]
[[Category:Caching]]

Revision as of 15:49, 7 May 2018

LVS and Varnish infrastructure

Wikipedia request flow

Diagram of "text" traffic flow through Wikimedia front edge LVS/nginx/Varnish infrastructure.

Naming

  • eqiad is representative of whichever data centre is currently primary (codfw is similar).
  • esams is representative of all caching sites (ulsfo is similar).
  • "upload" and "misc" traffic operate similarly to "text" shown here.

See Varnish and Clusters for more information about those in particular.

Routing

When LVS balances traffic to ports :80 (varnish), and :443 (nginx), it uses a hash of the client IP to help with TCP Fast Open and SSL session persistence respectively.

Within the caching layer (cp#xxx machines), the jump from nginx:443 to varnish:80 is direct on the local host.

However, the jump from varnish:80 (frontend) to varnish:3128 (backend) is different: for that jump, we hash on the URL (and other req meta-data) when balancing to the backends to divide the cache space among all machines, and thus the request typically moves from one machine to another within the same cluster.