You are browsing a read-only backup copy of Wikitech. The primary site can be found at

Portal:Toolforge/Admin/emergency guides/single tool webservice

From Wikitech-static
< Portal:Toolforge‎ | Admin‎ | emergency guides
Revision as of 18:08, 11 December 2017 by imported>Alangi derick (Tool Labs -> Toolforge)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

During the june 2015 NFS outage, we did an emergency deployment of the demw wiki loves earth jury tool on toolsbeta. This is a step-for-step guide what we did at that point, although some intermediate steps might be missing.

1) Find a suitable host.

  • We chose a host on Toolforge because it had on-line instances that could be deleted safely afterwards (spinning up new instances took an hour or more).
  • We chose a host with a suitable Security Group setup -- with port 8080 open to
  • We chose a host with a running web service setup so we could have an environment akin to the existing Toolforge one

Best host: toolsbeta-webgrid-04

2) Disable NFS and restart host (human intervention by Yuvi)

3) create basic structure in /data/project:

     /data/project/jury/public_html (containing salvaged code)

4) hack around the safeguards in lighttpd-starter

  • lighttpd-starter checks whether the server is allowed to start. It also tries to write in places where we don't have permission to write
--- /usr/local/bin/lighttpd-starter     2015-04-05 06:13:33.878601295 +0000
+++ lighttpd-starter   2015-07-03 21:45:21.945025980 +0000
@@ -23,14 +23,10 @@
-prefix=$(/bin/cat /etc/wmflabs-project)
+prefix=tools #$(/bin/cat /etc/wmflabs-project)
-home=$(getent passwd $user | cut -d : -f 6 | sed -e's/\/$//')
-if [ "$(getent group $user | cut -d : -f 1)" !="$user" ]; then
-  echo "$0: $1does not appear to be a tool" >&2
-  exit 1
+home=/data/project/jury #$(getent passwd $user | cut -d :-f 6 | sed -e 's/\/$//')
 if ["$home" = "" -o ! -d "$home/public_html" ]; then

5) start the webserver in a screen

sudo su tools.jury
script /dev/null
./lighttpd-starter jury 8080

6) Test if the webservice is running (ssh tunnel)

7) Disable webservice on Toolforge (requires root login)

  • stop webservice, make sure it stays stopped (no service.manifest, but that's hard without NFS)

8) set up proxy to redirect to temporary host

9) test on