Nova Resource:Integration/Setup

Roles

integration-agent-{type}-XXXX

Updated September 2019 based on https://phabricator.wikimedia.org/T226233

On https://horizon.wikimedia.org/project/instances/

• Create a new instance named integration-agent-{type}-XXXX where {type} is a role (example: docker) and XXXX increments starting from 1001.
• Source: pick the Stretch image
• Flavor: pick mediumram flavor

Wait a few minutes (during which the instance is created, initial setup happens). Then connect to the instance over SSH and fix puppet:

• sudo rm -fR /var/lib/puppet/ssl && sudo puppet agent -tv
• If that complains:
  • get the instance fully qualified domain name (FQDN): hostname --fqdn
  • On integration-puppetmaster-02.integration.eqiad.wmflabs, clean the old and invalid certificate(s): sudo puppet cert clean <FQDN OF INSTANCE HERE

Apply the Puppet role:

• https://horizon.wikimedia.org/project/instances/
• Click the instance then head to the tab Puppet
• Pick role::ci::slave::labs::docker
• For a Docker agent, set docker_lvm_volume variable to true.

Run puppet on the instance (puppet agent -tv) and verify:

• If a Docker agent, make sure there is a /var/lib/docker partition for Docker
• Clean unused packages: apt-get autoremove --purge
• Upgrade packages: apt-get -y dist-upgrade

Reboot the instance (Before adding to Jenkins). This cleans state, launches deamons, and fixes Shinken monitoring (phabricator:T91351). Once it is back, you can then add it to Jenkins

Add the instance to Jenkins

1. Create "New Node" in Jenkins management
   • Name: (short hostname of instance)
   • Type: Permanent Agent
   • Executors: 1 (for Docker agents: 4)
   • Remote root directory: /srv/jenkins/workspace
   • Labels:
     • contintLabsSlave
     • For Docker agents: Docker
   • Usage: EXCLUSIVE (Only build jobs with label restrictions matching this node)
   • Launch method: SSH
     • Host: (internal IP of instance)
     • Credentials: jenkins-deploy (key from role::ci::slave::labs::common)
   • Availability: Always (Keep this slave on-line as much as possible)

The Jenkins master will automatically trust the ssh key upon the first connection.

integration-dev

1. Create instance:
   • m1.medium
   • Security group: Default
2. Wait 10 minutes
3. Reconfigure instance from wikitech: Enable role::ci::slave::labs.
4. Via SSH, force a puppet run (applies role).

Utilities

puppet

Use sudo /usr/local/sbin/puppet-run &. Don't use sudo puppet agent -t, because that is not what cron uses and leads to inconsistencies with e.g. umask and other factors affecting default values used at runtime.