You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Postgres

From Wikitech-static
Revision as of 14:13, 9 December 2021 by imported>Hnowlan (→‎POSTGRES_HOT_STANDBY_DELAY: remove references to osm sync)
Jump to navigation Jump to search

Monitoring

Syncing Postgres replica

After a postgres slave replica has been re-imaged or it has fallen behind in replication due to network issues or other problems, it will first need to sync all data before it can be placed in production. The sre.postgresql.postgres-init cookbook can be used to resync a host

$ sudo cookbook sre.postgresql.postgres-init --replica ${master_server}  --reason "Resync after planet sync" --depool --pgversion 11

POSTGRES_HOT_STANDBY_DELAY

This alert indicates that a replica has fallen behind the master database in replication. This is not immediately a cause for panic, as a host can recover assuming it is not left out of replication for long. However, the postgres logs should be examined and connectivity between individual replicas and the master should also be checked.

If many or all hosts in the maps cluster are issuing this alert, it could be the case that the OSM dataset is being refreshed - during this operation it is likely that hosts will fall behind as large numbers of changes happen on the master during import. Check the cluster masters for import processes - generally this is a passing problem and will self-resolve over time. Follow the restart process if hosts need to be restarted

Related pages