You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Portal:Cloud VPS/Admin/notes/NAT loophole/NFS

From Wikitech-static
< Portal:Cloud VPS‎ | Admin‎ | notes/NAT loophole
Revision as of 15:43, 27 October 2020 by imported>Arturo Borrero Gonzalez (→‎idea 4: manila + LVM: reduce heading level)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Some ideas specific ideas about NFS.

Dumps share

  • Dumps is a RO share.
  • Data is being generated in the prod realm. There always will be a fundamental prod --> cloud data flow.
  • Dumps servers won't care what IP the client is using. RO means no write locks.
  • Potentially, cloud VMs could access dumps using the routing_source_ip NAT address (or floating IP). In either case, no need for dmz_cidr (i.e, no need for the dump server to know original VM address)

Tools share

Probably this applies to the scratch share as well. TODO: to maps too?

  • Share is RW.
  • Data is produced inside the cloud. There is no fundamental need for prod --> cloud or cloud --> prod data flows.
  • This section refers to the tools share, but might as well refers to the whole nfs::primary cluster.

Some ideas on how to address the tools share follows.

idea 1: serve NFS from a network namespace connected to a new cloud subnet

NFS.png
  • The NFS hardware servers get 2 NICs: one for control plane (ssh management, etc) and other for data plane (NFS).
  • The control plane interface is kept connected to the production network, for example the cloud-host vlan, which allows us to keep using install servers, puppet, etc
  • The data plane interface is attached to a linux network namespace. We then run the NFS daemon in this network namespace, thus adding an additional layer of isolation between prod and cloud realms.
  • We introduce a new NFS/share network, a physical vlan. The cloudgw device is the gateway for this new subnet. Let's call it cloud-nfs-share-subnet.
  • The netns in the NFS server is connected to this new cloud-nfs-share-subnet.
  • VM instances running in CloudVPS access the new cloud-nfs-share-submet directly without NAT.
  • The NFS connection is therefore never leaving the cloud realm.

idea 1: pros & cons

Pros:

  • leverages the cloudgw project.
  • incremental change without major service architecture reworks.
  • relatively low complexity from the engineering point of view.
  • labstore1004/1005 already have required NICs.

Cons:

  • this idea doesn't introduce openstack manila, i.e, we aren't introduce an upstream-cloud native component but a hand made setup.
  • doesn't leverage the ceph cluster

idea 2: introduce openstack manila using ceph as backend

manila is the openstack Shared Filesystems service for providing Shared Filesystems as a service. It can use the ceph storage backend we already have.

May require that VMs have direct access to the ceph cluster (for mounting the share). The ceph cluster is currently using private addressing in the prod network.

TODO: this idea requires more exploration.

idea 2: pros & cons

pros:

  • native openstack solution

cons:

  • VMs still need a raw NFS connection from the VM to the NFS server share. Manila doesn't solve any of this itself, it just provides the API for managing the share lifecycle and access control.

idea 3: introduce openstack manila using current NFS servers

We wrap current NFS behind the openstack manila API. This

idea 3: pros & cons

pros:

  • pure openstack solution


cons:

  • VMs still need a raw NFS connection from the VM to the NFS server share. Manila doesn't solve any of this itself, it just provides the API for managing the share lifecycle and access control.

idea 4: manila + LVM

TODO: elaborate