View source for Wm-bot

'''[[m:wm-bot|wm-bot]]''' is an IRC bot that participates in several developer-related wiki channels. wm-bot is currently running simultaneously on libera.chat

== All stars team ==
These folks are currently operating the bot, please insert your name if you are one of us:
* Petr Bena [[User:Petrb]] '''petan''' on Libera.chat (server access)
* Jeremyb [[User:jeremyb]] '''jeremyb''' on Libera.chat
* Billinghurst [[User:Billinghurst]] '''sDrewth''' on Libera.chat
* Thehelpfulone [[User:Thehelpfulone]] '''Thehelpfulone''' on Libera.chat
* Matthew Bowker [[User:Matthewrbowker]] '''Matthew_''' on Libera.chat (server access)
* MacFan4000 [[User:MacFan4000]] ''' MacFan4000 ''' on Libera.chat (server access)
* [[User:Legoktm]] '''legoktm''' on Libera.chat (server access)

== Where is it ==
Instance wm-bot.wm-bot.eqiad1.wikimedia.cloud, you need root to operate the bot.

* /opt/wm-bot (freenode)
* /opt/wm-bot-lib (libera)

== How does it work ==
Bot is using external bouncers (AKA IO buffer servers in its configuration). That means, you first start the bouncers, then you start restart script which keeps core running. The core is connected to bouncers, which are connected to network. In case you need to restart the core, bot stays connected to network.

This bot is using multiple instances but all are part of 1 huge system. The core handles all the instances but there is a bouncer for each of them.

<pre>
                                     ┌─────────────────────────┐
                                     │                         │
                                     │        Libera           │
                                     │                         │
                                     │                         │
                                     └─────────────────────────┘
                                          │         │
                                          │         │
                                          │         │
                                          │         │
                                          │         │
                                     ┌────┴────┐   ┌┴────────┐
                                     │         │   │         │
                                     │  BNC 1  │   │  BNC 2  │
                                     │         │   │         │
                                     └─────────┘   └─────────┘
                                         ▲             ▲
                                         │             │
                                         ├─────────────┘
                                         │
                             ┌───────────┴────┐
                             │                │
                             │    Bot core    │
                             │                │
                             └─┬─────────┬────┘
                               │         │
                               │         │
                               │         │
                               │         │
                               │         │
                               ▼         │
                          ┌───────┐      │                       ┌──────────────┐
                          │       │      └──────────────────────►│              │
                          │ MySQL │                              │ PostgreSQL   │
                          │       │                              │              │
                          └───────┘                              └──────────────┘

</pre>

== Paths ==

=== Freenode (historical, not in operation) ===

{| class="wikitable"
|-
! rowspan=2 | Binaries
| /opt/wm-bot/wmib.exe - core
|-
| /opt/wm-bot/bouncer.exe
|-
! Configuration
| /opt/wm-bot/configuration
|-
! Logs
| /opt/wm-bot/*.log
|-
! IRC Logs
| /var/www/html/logs
|}

=== Libera ===

{| class="wikitable"
|-
! rowspan=2 | Binaries
| /opt/wm-bot-lib/wmib.exe - core
|-
| /opt/wm-bot-lib/bouncer.exe
|-
! Configuration
| /opt/wm-bot-lib/configuration
|-
! Logs
| /opt/wm-bot-lib/*.log
|-
! IRC Logs
| /var/www/libera/libera_logs
|}

== How to start the bot ==
'''IMPORTANT''': If the server has crashed and bot wasn't properly shut down you have to remove all pid files using
 sudo rm -iv /mnt/share/wm-bot/*.pid

If bot isn't running you can start it using
 sudo service wm-bot start

=== Troubleshooting ===
First of all, check what is already running:
 ps -ef | grep wm-bot

If nothing is running, it should be ok, if some processes are there, kill them

* Ensure there are no *pid files in /mnt/share/wm-bot
* Make sure there is no restart.lock file
* There must be only 1 instance of core (restart.sh) running in same time

tail /mnt/share/wm-bot/wmib.log is your friend

== How to restart the bot core (fixes most problems) ==
This is what you should do if you just want to fix some bot issues, this will restart the core only, bot will stay connected to all channels and this process is painless and quick. Preferred:

* tail /mnt/share/wm-bot/wmib.log there must be no critical errors! If you see that bot couldn't write log files etc, do not kill it, or these data will be permanently lost!
* join #wm-bot
* type @restart

=== If you get permission denied ===
You forgot to put your cloak to admins file, just ssh to wm-bot instance and kill process running as wm-bot called "mono wmib.exe". This process will restart itself when it crashes.

== How to shutdown the bot ==

=== Complete shutdown ===
 sudo stop_all.sh

=== Core only ===
If you need to restart the bot because you changed the binaries and you need it to completely reload, you just need to execute "halt" or @restart in any channel. Keep in mind that doing restart after you changed the binary will likely fail (bot will crash instead of proper shutdown).

You can also shutdown the bot with signal (kill -15).

This command will also restart the bot, but it may not give it enough time to properly shut down (bot's proper shut down takes about 10 seconds)

 sudo service wm-bot restart

== How to fix 1 or more disconnected instances ==

Secret password is in /opt/wm-bot/configuration/admins (username is root) it's on first row third column

* Connect to local sysadmin console

$ telnet localhost 2820

* Now enter the username (root) and password

* Check the status (info / help)

<pre>
Successfuly logged in to wm-bot
info
Uptime: 0 days  0 hours since 7/18/2013 11:53:39 AM
Instances:
wm-bot channels: 50 connected: True working: True
wm-bot2 channels: 42 connected: True working: True
</pre>

If you see that some instance is not working properly, no matter of status, shut it down:

 kill wm-bot

Now, in second terminal, restart the corresponding bouncer, the bouncer is process bouncer.exe with parameter that correspond to broken instance, for example for wm-bot it's bouncer.exe 6667, list of all bouncers can be found at http://bots.wmflabs.org/~wm-bot/db/systemdata.htm

Once you kill that, you need start it again:

 nohup mono bouncer.exe 6667 &

Once the bouncer is running again you need to connect bot to it, type this in system console:

 conn wm-bot

== How to change the logs ==
Recommended:
In case you want to change logs, you must disable bot for that moment.

* Chmod the logfile with 000 (bot won't be able to write)
* Fix it and change the permissions back
* Connect to SQL database, using password in the <code>pg_pass=</code> variable in the <code>/opt/wm-bot/configuration/wmib.conf</code> file:

'''IMPORTANT''': PostgreSQL doesn't have any decent backups yet, so please be careful, damaging DB may not be reversible
<pre>
root@wm-bot2:~# psql -h wm-bot-pg2 -U wm_bot_lib

# Removing single line - this query may take minutes
> select * from logs where contents like '%text_you_search_for%';
> update logs set contents = 'redacted' where id = number_you_got_from_previous_query;
</pre>
* Drop the logs for the given channel (complete removal):
<pre>
wm_bot_lib=> delete from logs where channel='#mychannel';
</pre>

== How to patch the bot ==
 sudo ./easydeploy.sh

that will recompile all source code in ../wikimedia-bot stop the bot, replace the binaries and start the bot

== How to change the nickname of bot ==
Nickname that is chosen when bouncer reconnects to network is defined in configuration file (configuration/wmib.conf), each instance may have completely different nickname. If you want to change it only temporarily, or fix it back (in case nick is taken, bot will append random number to nick), you have two options:

* Wait - bot will automatically regain its hardcoded nick, it will perform check during periodic network resync, which happens about once a day (((Configuration.Channels.Count * 2000) + 80000) seconds to be precise).
* Fix it via telnet:
<pre>
petrb@wm-bot:~$ telnet localhost 2820
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Enter username:

Enter password:

Successfuly logged in to wm-bot, I have 1 users logged in
send wm-bot NICK wm-bot
send wm-bot2 NICK wm-bot2
quit
</pre>

Password can be found in configuration/admins file

== Backups ==

Are done using rsnapshot, see /etc/rsnapshot.conf, everything is stored under FS /backups, intervals are defined in /etc/cron.d/rsnapshot

[[Category:Bots]]