You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Maps: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Yurik
 
imported>Tfinc
Line 5: Line 5:


== Kartotherian ==
== Kartotherian ==
To thee the tiles without Varnish cache, connect to Kartotherian using an ssh tunnel, e.g. <code>ssh -L 6533:localhost:6533 maps-test2001.codfw.wmnet</code> and browse to http://localhost:6533  
To see the tiles without Varnish cache, connect to Kartotherian using an ssh tunnel, e.g. <code>ssh -L 6533:localhost:6533 maps-test2001.codfw.wmnet</code> and browse to http://localhost:6533  


== Tilerator ==
== Tilerator ==
Line 71: Line 71:
     sources: [{ref: v2}, {ref: gentmp}]
     sources: [{ref: v2}, {ref: gentmp}]
</syntaxhighlight>Once set, POST a job to copy <code>mixtmp</code> into the storage <code>v2</code> e.g. http://localhost:6534/add?generatorId=mixtmp&storageId=v2&baseZoom=0&fromZoom=5&beforeZoom=6&parts=10
</syntaxhighlight>Once set, POST a job to copy <code>mixtmp</code> into the storage <code>v2</code> e.g. http://localhost:6534/add?generatorId=mixtmp&storageId=v2&baseZoom=0&fromZoom=5&beforeZoom=6&parts=10
== Generating Tiles ==
Generate all tiles for zooms <code>0..7</code>, using generator <code>gen</code>, saving into <code>v3</code> everything including the solid tiles, up to 4 jobs per zoom.
http://localhost:6534/add?generatorId=gen&storageId=v3&parts=4&baseZoom=0&fromZoom=0&beforeZoom=8&saveSolid=1
Generated tiles only if they already exist in <code>v2</code> source, and save them into <code>v3</code>, on zooms <code>8..15</code>, 60 jobs per zoom.
http://localhost:6534/add?generatorId=gen&storageId=v3&parts=60&baseZoom=0&fromZoom=8&beforeZoom=16&sourceId=v2


== Bulk Copying ==
== Bulk Copying ==
The fastest way to copy a large number of tiles from one source to another is to use a large number of parts and specify <code>saveSolid=true</code> (skips solid tile detection). E.g. to copy all z16 tiles from v2 to v3, use<blockquote>http://localhost:6534/add?generatorId=v2&storageId=v3&zoom=16&parts=60&saveSolid=true </blockquote>
The fastest way to copy a large number of tiles from one source to another is to use a large number of parts and specify <code>saveSolid=true</code> (skips solid tile detection). E.g. to copy all z16 tiles from v2 to v3, use<blockquote>http://localhost:6534/add?generatorId=v2&storageId=v3&zoom=16&parts=60&saveSolid=true </blockquote>

Revision as of 23:32, 17 September 2015

Monitoring

Kartotherian

To see the tiles without Varnish cache, connect to Kartotherian using an ssh tunnel, e.g. ssh -L 6533:localhost:6533 maps-test2001.codfw.wmnet and browse to http://localhost:6533

Tilerator

Connect to Tilerator using an ssh tunnel, e.g. ssh -L 6534:localhost:6534 maps-test2001.codfw.wmnet

See full Tilerator documentation for all commands & parameters.

Command URL
Jobs Status http://localhost:6534/
Jobs Internal Status http://localhost:6534/raw
Get a list of variables http://localhost:6534/variables
Get source configuration http://localhost:6534/sources
Add new sources POST text body in YAML to http://localhost:6534/sources

For POST requests and responses in JSON, browser extensions such as Postman tend to work well.

Dynamic Tile Sources

Cassandra

To create a new Cassandra data source, post something like this to the /sources as a text body. Default table name is tiles. If table or keyspace is not there, you have to use createIfMissing parameter.

v2a:
  uri: cassandra://
  params:
    keyspace: v2
    table: tiles2a
    cp: [maps-test2001.codfw.wmnet, maps-test2002.codfw.wmnet, maps-test2003.codfw.wmnet, maps-test2004.codfw.wmnet]
    username: {var: cassandra-user}
    password: {var: cassandra-pswd}
#    repfactor: 4
#    durablewrite: 0
#    createIfMissing: true

Dynamic Layer Generator

To generate just a few layers from database, create a layer filter and a layer mixer:

gentmp:
  uri: bridge://
  xml:
    npm: ["osm-bright-source", "data.xml"]
  xmlSetDataSource:
    if:
      dbname: gis
      host: ""
      type: postgis
    set:
      host: localhost
      user: {var: osmdb-user}
      password: {var: osmdb-pswd}
  xmlLayers: [admin, road]

mixtmp:
  uri: layermixer://
  params:
    sources: [{ref: v2}, {ref: gentmp}]

Once set, POST a job to copy mixtmp into the storage v2 e.g. http://localhost:6534/add?generatorId=mixtmp&storageId=v2&baseZoom=0&fromZoom=5&beforeZoom=6&parts=10

Generating Tiles

Generate all tiles for zooms 0..7, using generator gen, saving into v3 everything including the solid tiles, up to 4 jobs per zoom.

http://localhost:6534/add?generatorId=gen&storageId=v3&parts=4&baseZoom=0&fromZoom=0&beforeZoom=8&saveSolid=1

Generated tiles only if they already exist in v2 source, and save them into v3, on zooms 8..15, 60 jobs per zoom.

http://localhost:6534/add?generatorId=gen&storageId=v3&parts=60&baseZoom=0&fromZoom=8&beforeZoom=16&sourceId=v2

Bulk Copying

The fastest way to copy a large number of tiles from one source to another is to use a large number of parts and specify saveSolid=true (skips solid tile detection). E.g. to copy all z16 tiles from v2 to v3, use

http://localhost:6534/add?generatorId=v2&storageId=v3&zoom=16&parts=60&saveSolid=true