Nova Resource:Integration/Setup

Roles

integration-slave-{type}-XXXX

1. Create instance:
   • ci1.medium
   • Security group: Default
     (Puppet master is auto-configured via Hiera:Integration)
2. Wait about 10 minutes (during which the instance is created, initial setup happens).
3. Connect to the instance over SSH.
4. Configure the server to use the integration-puppetmaster.
5. Reconfigure instance from wikitech: Enable role::ci::slave::labs.
6. Via SSH, force a puppet run (provisions the instance, takes about an hour).
7. You might have to force run puppet a few more times to complete provisioning
8. Once puppet is done, perform package upgrade: apt-get -y dist-upgrade
9. Reboot the instance (Before adding to Jenkins). This cleans state, launches deamons, and fixes Shinken monitoring (phabricator:T91351).
10. Create "New Node" in Jenkins management
    • Name: (short hostname of instance)
    • Type: Dumb
    • Executors: 1
    • Filesystem root: /mnt/jenkins-workspace
    • Labels:
      • contintLabsSlave
      • UbuntuTrusty phpflavor-hhvm phpflavor-php55
    • Usage: EXCLUSIVE (Only build jobs with label restrictions matching this node)
    • Launch method: SSH
      • Host: (internal IP of instance)
      • Credentials: jenkins-deploy (key from role::ci::slave::labs::common)
    • Availability: Always (Keep this slave on-line as much as possible)

integration-dev

1. Create instance:
   • m1.medium
   • Security group: Default
2. Wait 10 minutes
3. Reconfigure instance from wikitech: Enable role::ci::slave::labs.
4. Via SSH, force a puppet run (applies role).

Utilities

npm upgrade

Always use the original npm that came with Ubuntu to perform the upgrade. This installs the new version in /usr/local/bin, preserving the original.

• For each instance, gracefully depool it (!log it), wait for any running builds on it to complete, and run the following:
  • If Ubuntu Trusty:

# Note: Must sudo without preserving user environment, using -i
$ sudo -i
root$ /usr/share/npm/bin/npm-cli.js install -g npm@2.7.6
# Purge both caches 
root$ rm -rf /mnt/home/jenkins-deploy/.npm /root/.npm

• Re-pool instance.

puppet

Use sudo /usr/local/sbin/puppet-run &. Don't use sudo puppet agent -t, because that is not what cron uses and leads to inconsistencies with e.g. umask and other factors affecting default values used at runtime.

screenshot

# at integration-slave1010 over ssh
$ import -display :94 -window root "image $(date).png"

# local shell
$ scp integration-slave1010.eqiad.wmflabs:image*.png . && open

Debug MediaWiki frontend

Take slave of choosing offline in Jenkins web interface (e.g. press "Mark offline" after selecting a node)

sudo -iu jenkins-deploy
$ cd /mnt/jenkins-workspace/workspace/mediawiki-core-qunit/
$ export WORKSPACE=$PWD; export BUILD_TAG=debug; export EXECUTOR_NUMBER=999
$ cd src/
$ git remote update && git checkout origin/master && git reset --hard HEAD
$ /srv/deployment/integration/slave-scripts/bin/mw-install-mysql.sh
$ /srv/deployment/integration/slave-scripts/bin/mw-apply-settings.sh
$ . /srv/deployment/integration/slave-scripts/bin/mw-set-env-localhost.sh
$ echo -e "<?php\n\$wgServer = '${MW_SERVER}';\n\$wgScriptPath = '${MW_SCRIPT_PATH}';\n\$wgScript=\$wgStylePath=\$wgLogo=false;\n\$wgEnableJavaScriptTest = true;\n" >> "$MW_INSTALL_PATH/LocalSettings.php"
$ . /srv/deployment/integration/slave-scripts/bin/npm-setup.sh
$ rm -rf node_modules && npm install
$ ln -s "$MW_INSTALL_PATH" /srv/localhost-worker/$BUILD_TAG

Now the install is ready for interacting with.

For example:

$ curl --include "$MW_SERVER/$MW_SCRIPT_PATH/index.php?title=Special:BlankPage"
$ curl --include "$MW_SERVER/$MW_SCRIPT_PATH/load.php?debug=true&modules=startup&only=scripts"

Or

$ cd $MW_INSTALL_PATH
$ grunt qunit

Remember: Clean up MySQL database afterwards by running (needs the same environment, e.g. WORKSPACE, EXECUTOR_NUMBER):

/srv/deployment/integration/slave-scripts/bin/mw-teardown-mysql.sh