You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org
Maps: Difference between revisions
imported>Hnowlan |
imported>Jgiannelos No edit summary |
||
Line 4: | Line 4: | ||
[[File:Maps-components.png|thumb|Maps service component diagram]] | [[File:Maps-components.png|thumb|Maps service component diagram]] | ||
[[File:Maps-deployment.png|thumb|Maps service deployment diagram]] | [[File:Maps-deployment.png|thumb|Maps service deployment diagram]] | ||
[[File:Maps_@_PI@2x.png|thumb|Kartotherian internals focus]] | |||
The maps service consists of [https://github.com/kartotherian/kartotherian/blob/master/README.md Kartotherian] - a nodejs service to serve map tiles, [https://github.com/kartotherian/tilerator/blob/master/README.md Tilerator] - a non-public service to prepare vector tiles (data blobs) from OSM database into Cassandra storage, and TileratorUI - an interface to manage Tilerator jobs. There are four servers in the <code>maps</code> group: <code>maps-test200{1,2,3,4}.codfw.wmnet</code> that run Kartotherian (port 6533, NCPU instances), Tilerator (port 6534, half of NCPU instance), TileratorUI (port 6535, 1 instance). Also, there are four Varnish servers per datacenter in the <code>cache_maps</code> group. | The maps service consists of [https://github.com/kartotherian/kartotherian/blob/master/README.md Kartotherian] - a nodejs service to serve map tiles, [https://github.com/kartotherian/tilerator/blob/master/README.md Tilerator] - a non-public service to prepare vector tiles (data blobs) from OSM database into Cassandra storage, and TileratorUI - an interface to manage Tilerator jobs. There are four servers in the <code>maps</code> group: <code>maps-test200{1,2,3,4}.codfw.wmnet</code> that run Kartotherian (port 6533, NCPU instances), Tilerator (port 6534, half of NCPU instance), TileratorUI (port 6535, 1 instance). Also, there are four Varnish servers per datacenter in the <code>cache_maps</code> group. | ||
Revision as of 07:40, 9 October 2020
This page describes the technical aspects of deploying Maps service on Wikimedia Foundation infrastructure.
Intro
The maps service consists of Kartotherian - a nodejs service to serve map tiles, Tilerator - a non-public service to prepare vector tiles (data blobs) from OSM database into Cassandra storage, and TileratorUI - an interface to manage Tilerator jobs. There are four servers in the maps
group: maps-test200{1,2,3,4}.codfw.wmnet
that run Kartotherian (port 6533, NCPU instances), Tilerator (port 6534, half of NCPU instance), TileratorUI (port 6535, 1 instance). Also, there are four Varnish servers per datacenter in the cache_maps
group.
The infrastructure
Miscellaneous
Development processes
Puppetization and Automation
Prerequisites
- passwords and postgres replication configuration is set in Ops private repo (
root@palladium:~/private/hieradata/role/(codfw|eqiad)/maps/server.yaml
) - other configuration in
puppet/hieradata/role/(codfw|common|eqiad)/maps/*.yaml
cassandra::rack
is defined inpuppet/hieradata/hosts/maps*.yaml
- the
role::maps::master
/role::maps::slave
roles are associated to the maps nodes (site.pp)
Monitoring
- KPI dashboard
- Usage dashboard
- Usage - varnish
- Kartotherian - Grafana
- Kartotherian - Logstash
- Maps Cassandra - Logstash
- Tilerator - Grafana
- Tilerator - Logstash
Subpages
- Beta Cluster setup
- ClearTables
- ClearTables/Loading
- Debugging
- Dynamic tile sources
- External usage
- Infrastructure plans
- Kartotherian
- Kartotherian packages
- Keyspace Setup
- Maintenance
- Maps experiments
- OSM
- OSM Database Legacy
- Runbook
- Services deployment
- Tile storage
- Tilerator
- v2
- v2/Architecture
- v2/Common tasks
- v2/Infrastructure
- v2/Logs
- v2/Metrics
- v2/Troubleshooting