You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Nova Resource:Tools/SAL: Revision history

Jump to navigation Jump to search

Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: (cur) = difference with latest revision, (prev) = difference with preceding revision, m = minor edit.

(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)

16 February 2018

  • curprev 18:2118:21, 16 February 2018imported>Stashbot 199,148 bytes +937 arturo: upgrading tools-proxy-01 and tools-paws-master-01, same as others

15 February 2018

  • curprev 13:5413:54, 15 February 2018imported>Stashbot 198,211 bytes +672 arturo: cleanup ferm (deinstall) in tools-services-01 for T187435

14 February 2018

  • curprev 13:0913:09, 14 February 2018imported>Stashbot 197,539 bytes +236 arturo: the reboot was OK, the server seems working and kubectl sees all the pods running in the deployment (T187315)

11 February 2018

  • curprev 01:2801:28, 11 February 2018imported>Stashbot 197,303 bytes +367 zhuyifei1999_: `# find /home/ -maxdepth 1 -perm -o+w \! -uid 0 -exec chmod -v o-w {} \;` Affected: only /home/tr8dr, mode 0777 -> 0775

9 February 2018

  • curprev 10:3510:35, 9 February 2018imported>Stashbot 196,936 bytes +989 arturo: deploy https://gerrit.wikimedia.org/r/#/c/409226/ T179343 T182562 T186846

8 February 2018

  • curprev 18:3818:38, 8 February 2018imported>Stashbot 195,947 bytes +1,291 arturo: aborrero@tools-k8s-master-01:~$ sudo kubectl uncordon tools-worker-1002.tools.eqiad.wmflabs

6 February 2018

  • curprev 13:1513:15, 6 February 2018imported>Stashbot 194,656 bytes +302 arturo: deploy https://gerrit.wikimedia.org/r/#/c/408529/ to tools-services-01

5 February 2018

  • curprev 17:5817:58, 5 February 2018imported>Stashbot 194,354 bytes +325 arturo: publishing/unpublishing trusty-tools repo in tools-services-01 to address T186539

3 February 2018

  • curprev 01:0401:04, 3 February 2018imported>Stashbot 194,029 bytes +129 chicocvenancio: killed io intensive process in bastion-03 "vltools python3 ./broken_ref_anchors.py"

31 January 2018

29 January 2018

28 January 2018

  • curprev 22:4922:49, 28 January 2018imported>Stashbot 193,665 bytes +165 chicocvenancio: killed compromised session generating miner processes

27 January 2018

  • curprev 00:5500:55, 27 January 2018imported>Stashbot 193,500 bytes +209 arturo: at tools-static-11 the kernel OOM killer stopped git gc at about 20% :-(

25 January 2018

  • curprev 23:4723:47, 25 January 2018imported>Stashbot 193,291 bytes +422 arturo: fix last deprecation warnings in tools-elastic-03, tools-elastic-02, tools-proxy-01 and tools-proxy-02 by replacing by hand configtimeout with http_configtimeout in /etc/puppet/puppet.conf

23 January 2018

22 January 2018

  • curprev 18:3218:32, 22 January 2018imported>Stashbot 192,662 bytes +538 arturo: T181948 T185314 deploying jobutils and misctools v1.28 in the cluster

19 January 2018

18 January 2018

  • curprev 16:1116:11, 18 January 2018imported>Stashbot 191,768 bytes +877 arturo: aborrero@tools-clushmaster-01:~$ sudo aptitude purge vblade vblade-persist runit (for something similar to T182781)

17 January 2018

  • curprev 18:4818:48, 17 January 2018imported>Stashbot 190,891 bytes +692 arturo: aborrero@tools-clushmaster-01:~$ clush -w @all 'apt-show-versions | grep upgradeable | grep trusty-wikimedia' | tee pending-upgrades-report-trusty-wikimedia.txt

16 January 2018

  • curprev 22:0122:01, 16 January 2018imported>Stashbot 190,199 bytes +3,082 chasemp: qstat -explain E -xml | grep 'name' | sed 's/<name>//' | sed 's/<\/name>//' | xargs qmod -cq

11 January 2018

  • curprev 20:3320:33, 11 January 2018imported>Stashbot 187,117 bytes +848 andrewbogott: repooling tools-exec-1411, tools-exec-1440, tools-webgrid-lighttpd-1419, tools-webgrid-lighttpd-1420, tools-webgrid-lighttpd-1421

10 January 2018

  • curprev 15:1415:14, 10 January 2018imported>Stashbot 186,269 bytes +1,549 chasemp: tools-clushmaster-01:~$ clush -f 1 -w @k8s-worker "sudo puppet agent --enable && sudo puppet agent --test"

9 January 2018

  • curprev 23:2123:21, 9 January 2018imported>Stashbot 184,720 bytes +2,117 yuvipanda: paws new cluster master is up, re-adding nodes by executing same sequence of commands for upgrading

8 January 2018

  • curprev 20:3420:34, 8 January 2018imported>Stashbot 182,603 bytes +219 madhuvishy: Restart kube services and uncordon tools-worker-1001

6 January 2018

  • curprev 00:3500:35, 6 January 2018imported>Stashbot 182,384 bytes +1,399 madhuvishy: Run `clush -w @paws-worker -b 'sudo iptables -L FORWARD'`

4 January 2018

  • curprev 17:2417:24, 4 January 2018imported>Stashbot 180,985 bytes +120 andrewbogott: rebooting tools-paws-worker-1019 to verify repair of T184018

3 January 2018

31 December 2017

  • curprev 02:0102:01, 31 December 2017imported>Stashbot 180,671 bytes +102 bd808: Killed some pwb.py and qacct processes running on tools-bastion-03

21 December 2017

  • curprev 17:5717:57, 21 December 2017imported>Stashbot 180,569 bytes +199 bd808: PAWS: deleted hub-deployment pod stuck in crashloopbackoff

19 December 2017

18 December 2017

  • curprev 12:0412:04, 18 December 2017imported>Stashbot 180,173 bytes +621 arturo: it seems jupyterhub tries to use a database which doesn't exists: [E 2017-12-18 11:59:49.896 JupyterHub app:904] Failed to connect to db: sqlite:///jupyterhub.sqlite

15 December 2017

14 December 2017

  • curprev 16:5816:58, 14 December 2017imported>Stashbot 179,263 bytes +188 arturo: running clush -w @all 'sudo puppet agent --test' from tools-clushmaster-01.eqiad.wmflabs due to https://gerrit.wikimedia.org/r/#/c/394572/ being merged

13 December 2017

11 December 2017

  • curprev 19:3219:32, 11 December 2017imported>Stashbot 178,492 bytes +261 bd808: git gc on tools-static-11; --aggressive was killed by system (T182604)

1 December 2017

  • curprev 15:3315:33, 1 December 2017imported>Stashbot 178,231 bytes +218 chasemp: put the weird mess of untracked files on tools puppetmaster into stash to see what breaks as they should not be there?

30 November 2017

20 November 2017

  • curprev 20:3420:34, 20 November 2017imported>Stashbot 177,867 bytes +94 chasemp: backup crons tools-cron-01:/var/spool/cron# cp -Rp crontabs/ /root/20112017/
  • curprev 00:5200:52, 20 November 2017imported>Stashbot 177,773 bytes +128 andrewbogott: cherry-picking https://gerrit.wikimedia.org/r/#/c/392172/ onto the tools puppetmaster

17 November 2017

  • curprev 21:3321:33, 17 November 2017imported>Stashbot 177,645 bytes +232 valhallasw`cloud: also g-w'ed those files, and sent emails to all the affected users

16 November 2017

  • curprev 17:4017:40, 16 November 2017imported>Stashbot 177,413 bytes +291 chasemp: tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent --enable && sudo puppet agent --test && sudo unattended-upgrades -d'

15 November 2017

7 November 2017

  • curprev 01:2101:21, 7 November 2017imported>Stashbot 176,922 bytes +290 bd808: Removed all non-directory files from /home (via labstore1004 direct access)

5 November 2017

  • curprev 23:4823:48, 5 November 2017imported>Stashbot 176,632 bytes +391 bd808: Cleaned up 2 huge /tmp files left by tools.croptool (~6.5G)

3 November 2017

2 November 2017

1 November 2017

  • curprev 07:1107:11, 1 November 2017imported>Stashbot 176,084 bytes +213 madhuvishy: Clear nscd cache across all projects post labsdb dns switchover T179464

31 October 2017

  • curprev 16:5016:50, 31 October 2017imported>Stashbot 175,871 bytes +93 bd808: tools-bastion-03 (tools-login, login.tools) is overloaded

30 October 2017

  • curprev 17:3517:35, 30 October 2017imported>Stashbot 175,778 bytes +485 madhuvishy: Clear dns caches across tools hosts `sudo nscd -i hosts`
(newest | oldest) View ( | ) (20 | 50 | 100 | 250 | 500)