You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Proxy access to cluster"

From Wikitech-static
Jump to navigation Jump to search
imported>Yuvipanda
imported>Ayounsi
(Adding mention of a more restricted mgmt network)
Line 1: Line 1:
Presently we do not have any kind of VPN access to our cluster (or labs).  The preferred method by many Ops folks for accessing the web interfaces on these is via SSH -D option and [https://addons.mozilla.org/en-US/firefox/addon/foxyproxy-standard/ FoxyProxy].
This page document how to '''access web services on private hosts'''.


This document will outline how to setup this method of access. This method uses Firefox and the FoxyProxy addon linked above.
Presently, there is no VPN access to Wikimedia Foundation's production cluster. [[Portal:Cloud VPS|Cloud VPS]] does not provide this either.
 
== Tunnel ==
The below example will expose a web service from <code>analytics1001.eqiad.wmnet:8088</code> at your localhost as https://localhost:9088.<syntaxhighlight lang="bash">
ssh -N bast1002.wikimedia.org -L 9088:analytics1001.eqiad.wmnet:8088
</syntaxhighlight>The management network being restricted to <code>neodymium</code> and <code>sarin</code>, you can reach hosts with for example:
 
<code>ssh -L 8000:scs-eqsin.mgmt.eqsin.wmnet:443 neodymium.eqiad.wmnet</code>
 
Pointing then your web browser to:
 
https://localhost:8000, will actually show you (in this example) https://scs-eqsin.mgmt.eqsin.wmnet
 
If you need to reach an http port (eg. 80), don't forget to use http://localhost:8000 instead and for example <code>:80</code> in the ssh command.


* Install FireFox and FoxyProxy.
== FoxyProxy ==
:* Download the FoxyProxy ruleset [http://wikitech.wikimedia.org/files/wmf_foxyproxy_settings here].
The preferred method by Ops for accessing the web interfaces on private nodes is via the <code>-D</code> option of SSH in combination with [https://addons.mozilla.org/en-US/firefox/addon/foxyproxy-standard/ FoxyProxy].
:* In FoxyProxy settings, alternate click in the main window and select Import, then choose the wmf_foxyproxy_settings file you just downloaded.


* Setup your SSH -D sessions.
This document will outline how to setup this method of access.  This method uses Firefox and the FoxyProxy addon linked above.
:* You will need a session for each cluster you wish to access: production needs a connection, labs needs a connection
::* It is suggested you load up a screen session, create two new windows, and load each SSH session into its own windowTo do this run the following in terminal:
  screen
  ssh-add <path to your key for main cluster>
  ssh bast1001.wikimedia.org -D 8080 # or another bastion
  # Ctrl+a c (creates new screen window)
  ssh-add <path to your key for labs>
  ssh primary.bastion.wmflabs.org -D 8081 # or restricted.bastion for ops
  # Ctrl+a d (disconnects you from the screen sessions, allowing you to close or otherwise use terminal)


* Now when you load up the url patterns in the FoxyProxy settings, it will direct those URLs via your SSH tunnel to the correct cluster over the specified ports.
# Install FireFox and FoxyProxy.
* Please note that if you use OS X, you may want to 'exec ssh-agent bash' in your terminal screen sessions before adding your ssh key to ensure they are independently keyed and not shared across sessions.
#* Download the FoxyProxy ruleset [http://wikitech.wikimedia.org/files/wmf_foxyproxy_settings here].
#* In FoxyProxy settings, alternate click in the main window and select Import, then choose the wmf_foxyproxy_settings file you just downloaded.
# Setup an ssh -D session for the cluster(s) you need access to. (Repeat this as needed.)<source lang="bash">
screen
ssh bast1002.wikimedia.org -D 8080 # or another bastion
# Ctrl+a c (creates new screen window)
# Ctrl+a d (disconnects you from the screen session, allowing you to close or otherwise use terminal)</source>
# Now when you load up the url patterns in the FoxyProxy settings, it will direct those URLs via your SSH tunnel to the correct cluster over the specified ports.
# Please note that if you use OS X, you may want to 'exec ssh-agent bash' in your terminal screen sessions before adding your ssh key to ensure they are independently keyed and not shared across sessions.


The FoxyProxy URL patterns are listed below for ease of use, some are inclusive of others, but FoxyProxy in the past had some odd issues with them, so I find it best to just leave them all in the individual rulesets to eliminate potential issues.
The FoxyProxy URL patterns are listed below for ease of use, some are inclusive of others, but FoxyProxy in the past had some odd issues with them, so I find it best to just leave them all in the individual rulesets to eliminate potential issues.
Line 33: Line 44:
  ulsfo mgmt = *.mgmt.ulsfo.wmnet*
  ulsfo mgmt = *.mgmt.ulsfo.wmnet*
  ulsfo local = *.ulsfo.wmnet*
  ulsfo local = *.ulsfo.wmnet*
eqsin mgmt = *.mgmt.eqsin.wmnet*
eqsin local = *.eqsin.wmnet*
  labs = *.eqiad.wmflabs*
  labs = *.eqiad.wmflabs*
[[Category:How-To]]
 
== See also ==
* [[Production shell access#SSH configuration]]
 
[[Category:How-To]]

Revision as of 19:36, 11 July 2018

This page document how to access web services on private hosts.

Presently, there is no VPN access to Wikimedia Foundation's production cluster. Cloud VPS does not provide this either.

Tunnel

The below example will expose a web service from analytics1001.eqiad.wmnet:8088 at your localhost as https://localhost:9088.

ssh -N bast1002.wikimedia.org -L 9088:analytics1001.eqiad.wmnet:8088

The management network being restricted to neodymium and sarin, you can reach hosts with for example:

ssh -L 8000:scs-eqsin.mgmt.eqsin.wmnet:443 neodymium.eqiad.wmnet

Pointing then your web browser to:

https://localhost:8000, will actually show you (in this example) https://scs-eqsin.mgmt.eqsin.wmnet

If you need to reach an http port (eg. 80), don't forget to use http://localhost:8000 instead and for example :80 in the ssh command.

FoxyProxy

The preferred method by Ops for accessing the web interfaces on private nodes is via the -D option of SSH in combination with FoxyProxy.

This document will outline how to setup this method of access. This method uses Firefox and the FoxyProxy addon linked above.

  1. Install FireFox and FoxyProxy.
    • Download the FoxyProxy ruleset here.
    • In FoxyProxy settings, alternate click in the main window and select Import, then choose the wmf_foxyproxy_settings file you just downloaded.
  2. Setup an ssh -D session for the cluster(s) you need access to. (Repeat this as needed.)
    screen
    ssh bast1002.wikimedia.org -D 8080 # or another bastion
    # Ctrl+a c (creates new screen window)
    # Ctrl+a d (disconnects you from the screen session, allowing you to close or otherwise use terminal)
    
  3. Now when you load up the url patterns in the FoxyProxy settings, it will direct those URLs via your SSH tunnel to the correct cluster over the specified ports.
  4. Please note that if you use OS X, you may want to 'exec ssh-agent bash' in your terminal screen sessions before adding your ssh key to ensure they are independently keyed and not shared across sessions.

The FoxyProxy URL patterns are listed below for ease of use, some are inclusive of others, but FoxyProxy in the past had some odd issues with them, so I find it best to just leave them all in the individual rulesets to eliminate potential issues.

Some of the patterns I use are inclusive of the others, mostly because I happened to setup mgmt and didn't bother with non-mgmt for awhile. So you could always clean it up and remove the stuff that is duplication due to pattern matching:

eqiad mgmt = *.mgmt.eqiad.wmnet*
eqiad local = *.eqiad.wmnet*
esams mgmt = *.mgmt.esams.wmnet*
esams local = *.esams.wmnet*
codfw mgmt = *.mgmt.codfw.wmnet*
codfw local = *.codfw.wmnet*
ulsfo mgmt = *.mgmt.ulsfo.wmnet*
ulsfo local = *.ulsfo.wmnet*
eqsin mgmt = *.mgmt.eqsin.wmnet*
eqsin local = *.eqsin.wmnet*
labs = *.eqiad.wmflabs*

See also