You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Tool:CommTech

From Wikitech-static
Revision as of 00:46, 17 July 2019 by imported>MusikAnimal (lang="bash")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Toolforge tools
Crystal Clear app package utilities.png CommTech
Website https://commtech.wmflabs.org
Description Hosts Community Tech's test wiki and bot
Maintainer(s) Matthewrbowker, MusikAnimal, Samwilson (View all)
License MIT License
Issues Open tasks · Report a bug

This page is for documentation related to the commtech VPS project.

CommTech Wiki

CommTech Wiki is a MediaWiki-Vagrant instance used for internal testing of MediaWiki features. It lives on the VPS instance commtech-2.commtech.eqiad.wmflabs.

Community Tech bot

This section documents bot tasks that live on the commtech-bot.commtech.eqiad.wmflabs instance.

Creating a new instance

  1. Create a new instance (m1.medium or larger)
  2. Install PHP along with some dependencies (using a Ondřej Surý Debian package).
    sudo apt-get install apt-transport-https lsb-release ca-certificates
    sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
    echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list
    sudo apt-get update
    sudo apt-get install -y php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-json
    sudo apt-get install -y zip unzip php7.2-zip
    
  3. Install composer by following these instructions, but make sure to install to the /usr/local/bin directory and with the filename composer, e.g.:
    sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer
    
  4. Create the bot directory with sudo mkdir /var/bot
  5. Create the commtech-bot user and give it ownership of the bot directory.
    sudo useradd -M commtech-bot 
    sudo usermod -L commtech-bot
    sudo chown commtech-bot:commtech-bot bot
    

Popular Pages bot

After creating an instance (see above), do the following:

  1. Clone the repo in the bot directory:
    sudo su commtech-bot # Make sure you're the bot user
    cd /var/bot
    git clone https://github.com/wikimedia/popularpages.git
    
  2. Follow the installation steps at https://github.com/wikimedia/popularpages
  3. Edit the crontab for commtech-bot with crontab -e then add the following to the bottom:
    */30 * 3-18 * * /usr/bin/flock -n /tmp/fcj.lockfile /var/bot/popularpages/checkReports.php en.wikipedia
    
    Using flock ensures there's only ever one instance of the bot running at a time.

Note that you can run commands as a different user. This will make it easier for instance if you need to simply pull in the latest changes, e.g. runuser -l commtech-bot -c 'git pull origin master'.

NSFW

The NSFW instance is at commtech-nsfw.commtech.eqiad.wmflabs. This is a NSFW classifier service, using Yahoo!'s open source algorithm, wrapped in a Docker container.

See https://github.com/rahiel/open_nsfw-- for installation instructions.

To query the service, make a POST request with cURL:

$ curl -d 'url=https://upload.wikimedia.org/wikipedia/commons/thumb/b/b3/Jimmy_Wales_in_August_2006.jpg/319px-Jimmy_Wales_in_August_2006.jpg' https://nsfw.wmflabs.org
0.006064464803785086