LVS and Varnish

Wikimedia infrastructure

Data centers

Networking

Global traffic routing

• Global traffic routing
• Caching overview
• DNS
• HTTPS
• HAProxy
• Varnish
• Apache Traffic Server
• Traffic cache hardware
• HTTP timeouts
• Purged
• MediaWiki HTTP cache headers
• Kafka HTTP purging
• URI Path Normalization

MediaWiki

Multimedia

Data Persistence

Observability

Search

Data Engineering

[]

LVS and Varnish infrastructure

Wikipedia request flow

Naming

• eqiad is representative of whichever data centre is currently primary (codfw is similar).
• esams is representative of all caching sites (ulsfo is similar).

• "upload" traffic operates similarly to "text" shown here.

See Varnish and Clusters for more information about those in particular.

Routing

When LVS balances traffic to ports :80 (varnish), and :443 (nginx), it uses a hash of the client IP to help with TCP Fast Open and SSL session persistence respectively.

Within the caching layer (cp#xxx machines), the jump from nginx:443 to varnish:80 is direct on the local host.

However, the jump from varnish:80 (frontend) to varnish:3128 (backend) is different: for that jump, we hash on the URL (and other req meta-data) when balancing to the backends to divide the cache space among all machines, and thus the request typically moves from one machine to another within the same cluster.