You are browsing a read-only backup copy of Wikitech. The live site can be found at

Maps/v2/Common tasks

From Wikitech-static
< Maps‎ | v2
Revision as of 23:06, 1 October 2021 by imported>Quiddity (lang=bash)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Common maps tasks

Access map nodes

ssh maps1009.eqiad.wmnet

Deploy new code


Currently, kartotherian main code is available in the imposm branch.

In order to deploy kartotherian you need to package the deploy repo locally and submit to gerrit, kartotherian uses service-runner to run a Docker container for packaging, the following script assumes you have both the package repo and the deploy repo at the same directory level:

cd ../package
git status
git reset --hard origin/imposm
git checkout .
git clean -fd
git checkout imposm
git pull origin imposm
git status
git --no-pager log --decorate -n 1
rm -rf node_modules/
npm i service-runner

cd ../deploy
git status
git reset --hard origin/imposm
git checkout .
git clean -fd
git checkout imposm
git pull
git --no-pager log --decorate -n 1
git submodule update --init
git status

cd ../package
./server.js build --deploy-repo --force

Configuration changes


The tegola config is maintained using helm in the deployment charts repo and populated by the helm values per env:


Kartotherian is using scap for deployment. The configuration is maintained here:


Postgres config is maintained using puppet. Here are the relevant sections:


OSM import process is configured using puppet. Here are the relevant sections:


Enable/disable import process

In order to disable the import process you need to switch the boolean value to false and run puppet:

  • profile::maps::osm_master::disable_replication_cron: false

Run a planet import

When running a planet import, make sure that the following checklist is followed:

  • disable disable_replication_service and disable_tile_generation_cron in hieradata
  • execute initial-imposm-import script using the following command
# change -d accordingly when performing the import with the most updated values
imposm-initial-import -d 210906 -x webproxy.eqiad.wmnet:8080
  • monitor the full planet import
    • Is there any disk space issue?
    • Did the script finished properly?
    • Are the logs sufficient?
  • monitor and check the APIs, if needed rollback with the following command
sudo -u osmupdater imposm-rollback-import
  • re-enable the replication service and check for its health
  • re-enable the tile-generation cron and check for its health
  • remove imposm backup to free disk space
sudo -u osmupdater imposm-removebackup-import

Pooling/depooling nodes


Get a DB shell

> ssh maps1005.eqiad.wmnet

> sudo su

> su - postgres

> psql

postgres=# \c gis

Run an example query

Select all landuse geometries included in the bounding box:

  • SRID 3857 (the one we use in our layers config)
  • Box: left=2632692 bottom=4572168 right=2648887 top=4581940

This bounding box is a rectangle that includes the center of Athens, Greece.

gis=# SELECT osm_id, ST_AsMVTGeom(geometry, ST_MakeEnvelope(2632692,4572168,2648887,4581940, 3857)) AS geom FROM layer_landuse(ST_MakeEnvelope(2632692,4572168,2648887,4581940, 3857), 9);

Reindex DB

WARNING this operation might take even ~10 hours to be completed

From a pgsql shell, connect to the `gis` DB and run:


Example HTTP requests


Get the raster tile x=0 y=0 z=0 for source osm-intl curl


Tegola currently is only used internally. In order to reach out for the current example we will ssh to the deployment node (eq. in eqiad)

> ssh deployment.eqiad.wmnet

To get the endpoint information for different maps and layers:

> curl https://tegola-vector-tiles.svc.eqiad.wmnet:4105/capabilities | jq .

To get the vector tile x=0 y=0 z=0 for map osm

> curl https://tegola-vector-tiles.svc.eqiad.wmnet:4105/maps/osm/{z}/{x}/{y}.pbf


Post an event:

curl -X POST -d '
    "\$schema": "/maps/tile_change/1.0.0",
    "meta": {
    "dt": "$(date --iso-8601=s)"
    "stream": "maps.tile_expiration"
"tile": "1/0/2",
"state": "expired"
}' https://eventgate-main.svc.eqiad.wmnet:4492/v1/events/

Example tegola commands

  • Run server
    • tegola -c </path/to/config> serve
  • Cache a tile
    • tegola -c /path/to/config cache seed tile-name 0/0/0
  • Purge a tile from cache
    • tegola -c /path/to/config cache purge tile-name 0/0/0