You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Incidents/2021-09-26 appserver latency

From Wikitech-static
< Incidents
Revision as of 17:49, 8 April 2022 by imported>Krinkle (Krinkle moved page Incident documentation/2021-09-26 appserver latency to Incidents/2021-09-26 appserver latency)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

document status: in-review

Summary

Increased db load for enwiki (s1) resulted in slower responses, which in turn resulted in overall php-fpm worker limits being reached and thus affecting requests for all wikis. For requests above the limit, the error was "upstream connect error or disconnect/reset before headers. reset reason: overflow".

Impact: For about 15 minutes, backend appservers were slower or unable to respond for all wikis. This mainly affected logged-in users and most bot/API queries. Some page views from unregistered users were affected, for pages that were recently edited or otherwise expired from the CDN cache.

Documentation:

Actionables