You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Zedler/Replication

From Wikitech-static
Jump to navigation Jump to search

The replication is done with 2 different programs. At first with the normal mysql-replication-program and at secound with a tool programmed by river (trainwreck). The reason for this is, that mysql can only replicated one master at the same time.

So the mysql-replication is used to replicate the s2-cluser (dewiki and a few others) on zedler, and s1 (enwiki) on yarrow. The replication tool replicates s3 (frwiki and the rest) on yarrow.

The connection to the masters in Florida is done by a few ssh-tunnels. These ssh-tunnels are started by a little-bash-script /usr/local/sbin/sqltunnel". The tunnels listen at different ports at zedler and forward the traffic to different masters in Florida. 330X is the port-schema at zedler. X stand for the cluster-number (s1, s2 and s3). See the table:

Cluster Wikis Port at
zedler
Replication-
program
s1 enwiki 3301 MySQL
s2 dewiki *1 3302 MySQL
s3 frwiki *2 3303 trainrweck

*1: and a few other wikis
*2: and the rest

Password

The mysql-user-user and -password, that are used for replication, can find at "/root/repl". Never give it to anyone!

MySQL-Replication

The MySQL-Replication is done normaly by mysql. Mysql listen to localhost and port 3302. Replication can be start with "slave start;" and stop with "slave stop;" at the mysql-bash. For the logfile-position see here. More informations about mysql-replication can find at [1].

DaB.s Replication-Program

DaBs programm can be find at "/root/syncdb/". Normaly, each instance runs in a screen and each instance can be start with "startit.s1" for s1 or "startit.s3" for s3. That starts a little bash-script that runs the java-programm in a loop. The logfiles can be find at "/root/syncdb/logs/"; the config-files can find at "/root/syncdb/config/". The programm is work-in-progress.

Howto to restart replication after the master-db-server changed

  1. Ask the wikimedia-server-admins for the cluster, the new mastername, the new logfile-name and the new logfileposition.

Stopping

  1. Login to zedler
  2. Login to mysql.
  3. (run "/bin/bash" if you like).
  4. Run "slave stop;" to stop mysql-replication
  5. Leave mysql with "quit;" (difficult, isn't it? ;))
  6. Run "ps -ef|grep startit.s" for searching for all dabs-replication-program-loops
  7. Kill all dabs-replication-program-loops with "kill -9 processid processid processid....".
  8. Run "ps -ef|grep syncdb" for searching for all dabs-replication-programs
  9. Kill all dabs-replication-programs with "kill -9 processid processid processid....".
  10. Run "ps -ef|grep sqltunnel" for searching the sql-tunnel-script.
  11. Kill all sql-tunnel-processes with "kill -9 processid processid processid....".
  12. Run "ps -ef|grep ssh|grep 3306" for searching the ssh-tunnels.
  13. Kill all sql-tunnels with "kill -9 processid processid processid....".

Changing

  1. Open "/usr/local/sbin/sqltunnel" with your favorite editor.
  2. Change the startcommands. For example: Change "start_tunnel X 3301" to "start_tunnel Y 3301" for a masterchange from X to Y.
  3. Save the file.

Starting

  1. Run the sql-tunnel-script with "nohup /usr/local/sbin/sqltunnel&".
  2. Login to mysql again.
  3. If s2 was changed: Change the logfile-name and the logfileposition for mysql. For example, if the name changed to Y-bin.001 and the position to 4, then run "CHANGE MASTER TO MASTER_LOG_FILE='Y-bin.001', MASTER_LOG_POS=4;". See [2] for more details.
  4. Start mysql-replication with "slave start;"
  5. If s1 or s3 was changed: Change the db with "use sync_34176;". Then update the position with "UPDATE pos SET logdatei="logfilename", logpos=logfilepos where pos_id=X". X is 1 or 3 (1 for s1 and s3). Be very sure, what X you use!
  6. Leave mysql with "quit;"
  7. Open a screen with "screen"
  8. Run "/bin/bash"
  9. Goto "/root/syncdb"
  10. Start s3-replication with "./startit.s3.sh"
  11. Leave the screen with "Controll+a and d" (d without controll-pressed)
  12. Repeat the last 5 steps for s1 (replace all s3 with s1)
  13. Be happy :)