You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

PyBal

From Wikitech-static
Revision as of 07:38, 12 September 2014 by imported>Giuseppe Lavagetto (Update the location of pybal config files, correct some very outdated info.)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

PyBal is a LVS monitoring script quite similar to lvsmon. It's written in Python using the Twisted framework.

For more information about Wikimedia's LVS setup in general, see LVS.

At this moment, just a few features distinguish it from lvsmon:

  • It's using asynchronous communication, and thus runs all checks in parallel instead of sequentially
  • It has an extra monitoring method called IdleConnection, which keeps an idle connection open to all squids, and therefore notices immediately when the Squid processes are shut down / crashing
  • It can fetch server lists over HTTP as well as from the local filesystems

...but I intend to polish it more, and extend it with useful things.

The script is in Wikimedia's Git repo "operations/debs", subdirectory pybal.

-- Mark

Setup

PyBal is currently installed on our LVS hosts, in directory /usr/sbin. Start or stop it by

 /etc/init.d/pybal start|stop

Configuration is in /etc/pybal/. pybal.conf defines the LVS service parameters

The list of pooled hosts resides wherever the =pybal::web= class is installed via puppet, under the directory =/srv/pybal-config= and it will be reachable via the internal address =http://configuration-master.$site.wmnet/pybal=, with one file per LVS service. Attributes:

  • weight: a larger number means that more requests get sent to this server in comparison with others
  • enabled: either True or False, depnding on whether you want requests to be sent to this server

The format should be fairly self explanatory; the files more or less use Python assignment / dictionary syntax.

PyBal supports multiple LVS services through a single instance and configuration file pybal.conf, e.g.:

[text]
protocol = tcp
ip = 145.97.39.155
port = 80
scheduler = wlc
config = file:///etc/pybal/text-squids

[images]
protocol = tcp
ip = 145.97.39.156
port = 80
scheduler = wlc
config = file:///etc/pybal/upload-squids

Beware, the code as checked out from git has DryRun = True set in ipvs.py, meaning that it will not modify any actual IPVS state but only show the commands for debugging. This should be changed to a commandline option, but for now edit that file to DryRun = False.

The configuration files are generated via puppet.

Howto

See LVS.