You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Deployment server: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Dzahn
(Created page with "Deployment servers are servers used for deployment. As of May 2019 the deployment servers are: deploy1001 in eqiad and deploy2001 in codfw. There is also...")
 
imported>Legoktm
(→‎Service: rv, deploy1002 is still active deployment server)
 
(6 intermediate revisions by 6 users not shown)
Line 1: Line 1:
Deployment servers are servers used for [[deployment]].
'''Deployment servers''' are servers used for [[deployment]] of software using [[Scap]] and [[scap3]].


As of May 2019 the deployment servers are: [[deploy1001]] in [[eqiad]] and [[deploy2001]] in [[codfw]].
== Service ==
As of June 2021 the deployment servers are:
* [[deploy1002]], currently active '''primary'''
* [[deploy2002]], passive/spare


There is also a service alias [[deployment.eqiad.wmnet]] pointing to the current server.
There is also a service alias <code>deployment.eqiad.wmnet</code> which always points to the currently active deployment (Yes, this shortcut does the right thing, despite having "eqiad" in its name).


A former deployment server was [[tin]].
== Runbooks ==
=== How to setup a deployment server in a cloud VPS project ===
* create a new cloud VPS instance of size m1.small (or m1.medium if you will be deploying many scap3 projects)
* in [[Horizon]] apply the puppet 'role::deployment_server' on the instance
* [[Help:Adding Disk Space to Cloud VPS instances|Create and attach]] an extra storage volume on <code>/srv</code> to hold the software you are going to deploy
* in the operations/puppet repo in ./hieradata/labs/''yourproject''/common.yaml or in Horizon add Hiera key/value pairs:
 
<pre>
has_lvs: false
has_tls: false
mcrouter::has_ssl: false
mcrouter::port: 11213
mcrouter::shards:
  wancache:
    eqiad:
      shard01:
        host: deployment-memc04
        port: 11211
      shard02:
        host: deployment-memc05
        port: 11211
      shard03:
        host: deployment-memc08
        port: 11211
profile::kubernetes::deployment_server::git_group: wikidev
profile::kubernetes::deployment_server::git_owner: trebuchet
profile::mediawiki::apc_shm_size: 128M
profile::mediawiki::deployment::server::rsync_host: deploy1001.devtools.eqiad1.wikimedia.cloud
profile::mediawiki::php::enable_fpm: true
profile::mediawiki::php::php_version: '7.2'
profile::rsyslog::kafka_shipper::kafka_brokers: []
profile::services_proxy::ensure: absent
profile::services_proxy::services:
  dummy:
    hostname: dummy.example.net
    localport: 9999
    port: 9999
    scheme: https
    timeout: 600
redis::shards:
  sessions:
    eqiad:
      shard01:
        host: 172.16.5.76
        port: 6379
      shard02:
        host: 172.16.5.17
        port: 6379
      shard03:
        host: 172.16.5.12
        port: 6379
      shard04:
        host: 172.16.5.2
        port: 6379
scap::dsh::groups:
  librenms:
    hosts:
    - deployment-netbox.deployment-prep.eqiad1.wikimedia.cloud
  mediawiki-api-canaries:
    hosts: []
  mediawiki-appserver-canaries:
    hosts:
    - deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
  mediawiki-installation:
    hosts:
    - deployment-jobrunner03.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-09.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-snapshot01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-deploy01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-deploy02.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mwmaint01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-parsoid10.deployment-prep.eqiad1.wikimedia.cloud
  restbase-betacluster:
    hosts:
    - deployment-restbase01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-restbase02.deployment-prep.eqiad1.wikimedia.cloud
  webperf:
    hosts:
    - deployment-webperf11.deployment-prep.eqiad1.wikimedia.cloud
scap::sources:
  gerrit/gerrit:
    repository: operations/software/gerrit
  gervert/deploy:
    repository: operations/software/gerrit/tools/gervert/deploy
  phabricator/deployment:
    repository: phabricator/deployment
</pre>
 
* adjust the ''deployment::server::rsync_host'' value to a host in your project
* adjust ''scap::sources'' to contain the scap3 projects you will be deploying
* run puppet (a couple times)
* run sudo -u trebuchet /usr/bin/scap deploy --init inside each of the repos under /srv/deployment/, so for example in /srv/deployment/gerrit/gerrit and so on but NOT in /srv/deployment/ directly.
* run git init in /srv/deployment
* run puppet again
 
== See also ==
 
* [[Maintenance server]]
 
* [[User:BryanDavis/Scap3 in a Cloud VPS project]]

Latest revision as of 23:56, 29 June 2021

Deployment servers are servers used for deployment of software using Scap and scap3.

Service

As of June 2021 the deployment servers are:

There is also a service alias deployment.eqiad.wmnet which always points to the currently active deployment (Yes, this shortcut does the right thing, despite having "eqiad" in its name).

Runbooks

How to setup a deployment server in a cloud VPS project

  • create a new cloud VPS instance of size m1.small (or m1.medium if you will be deploying many scap3 projects)
  • in Horizon apply the puppet 'role::deployment_server' on the instance
  • Create and attach an extra storage volume on /srv to hold the software you are going to deploy
  • in the operations/puppet repo in ./hieradata/labs/yourproject/common.yaml or in Horizon add Hiera key/value pairs:
has_lvs: false
has_tls: false
mcrouter::has_ssl: false
mcrouter::port: 11213
mcrouter::shards:
  wancache:
    eqiad:
      shard01:
        host: deployment-memc04
        port: 11211
      shard02:
        host: deployment-memc05
        port: 11211
      shard03:
        host: deployment-memc08
        port: 11211
profile::kubernetes::deployment_server::git_group: wikidev
profile::kubernetes::deployment_server::git_owner: trebuchet
profile::mediawiki::apc_shm_size: 128M
profile::mediawiki::deployment::server::rsync_host: deploy1001.devtools.eqiad1.wikimedia.cloud
profile::mediawiki::php::enable_fpm: true
profile::mediawiki::php::php_version: '7.2'
profile::rsyslog::kafka_shipper::kafka_brokers: []
profile::services_proxy::ensure: absent
profile::services_proxy::services:
  dummy:
    hostname: dummy.example.net
    localport: 9999
    port: 9999
    scheme: https
    timeout: 600
redis::shards:
  sessions:
    eqiad:
      shard01:
        host: 172.16.5.76
        port: 6379
      shard02:
        host: 172.16.5.17
        port: 6379
      shard03:
        host: 172.16.5.12
        port: 6379
      shard04:
        host: 172.16.5.2
        port: 6379
scap::dsh::groups:
  librenms:
    hosts:
    - deployment-netbox.deployment-prep.eqiad1.wikimedia.cloud
  mediawiki-api-canaries:
    hosts: []
  mediawiki-appserver-canaries:
    hosts:
    - deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
  mediawiki-installation:
    hosts:
    - deployment-jobrunner03.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-09.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-07.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-snapshot01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-deploy01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-deploy02.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mwmaint01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-mediawiki-parsoid10.deployment-prep.eqiad1.wikimedia.cloud
  restbase-betacluster:
    hosts:
    - deployment-restbase01.deployment-prep.eqiad1.wikimedia.cloud
    - deployment-restbase02.deployment-prep.eqiad1.wikimedia.cloud
  webperf:
    hosts:
    - deployment-webperf11.deployment-prep.eqiad1.wikimedia.cloud
scap::sources:
  gerrit/gerrit:
    repository: operations/software/gerrit
  gervert/deploy:
    repository: operations/software/gerrit/tools/gervert/deploy
  phabricator/deployment:
    repository: phabricator/deployment
  • adjust the deployment::server::rsync_host value to a host in your project
  • adjust scap::sources to contain the scap3 projects you will be deploying
  • run puppet (a couple times)
  • run sudo -u trebuchet /usr/bin/scap deploy --init inside each of the repos under /srv/deployment/, so for example in /srv/deployment/gerrit/gerrit and so on but NOT in /srv/deployment/ directly.
  • run git init in /srv/deployment
  • run puppet again

See also