You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
News/CloudVPS NAT wikis
This page is currently a draft.
More information and discussion about changes to this draft on the talk page.
This page describes a network change in the CloudVPS service, which affects how processes running inside the cloud will reach WMF-hosted wiki services, including wikidata, commons, etc.
What is changing ?
The CloudVPS network has a general egress NAT public IPv4 address, known as
This public address is used to translate internal virtual machine private addresses, which look like
However, traditionally, there was a networking policy exception that prevented the private ranged from being translated when the destination of the network connection was a WMF-hosted wiki. This means that, previous to this change, WMF-hosted wikis would see the internal virtual machine private address.
The change covered in this page is precisely to drop this networking exception, so WMF-hosted wikis will see network traffic originated from CloudVPS as coming from the general NAT address.
The following diagrams may help visualize what is changing.
Before this change:
After this change:
TODO: double check this with the team.
- 2021-01-25: announce the change to the community. Ask for feedback.
- 2021-02-25: evaluate collected feedback, address concerns, work with affected stakeholders towards a smooth change.
- 2021-03-25: introduce change. Monitor services to discover bugs and other issues, and fix them.
- 2021-03-31: change is considered done and completed.
What should I do?
TODO: is wiki admin the right term?
If you are a CloudVPS project owner / users
If your virtual machine instances contact WMF-hosted wikis in any way, be inform of this change. You don't have do to anything specific other than monitor that your services keep working as expected.
In case you detect your service no longer works due to a WMF-hosted wiki block, ratelimit or similar, please contact the WMCS team or the Wiki admin directly.
If you are a Toolforge developers / user
It is very likely that your Toolforge tool interact in some way with WMF-hosted wikis. You don't have to do anything specific other than monitor that your tool keep working as expected.
In case you detect your tool no longer works due to a WMF-hosted wiki block, ratelimit or similar, please contact the WMCS team or the Wiki admin directly.
If you are a Wiki admin
There will be a bunch of new connections to your wiki coming from
184.108.40.206. Review rate limits, blocks and tools/bots exceptions to account for this new address.
Solutions to common problems
Why are we doing this?
There are several technical reasons that suggest this change should be done as soon as possible.
One of the most important ones is realm separation. WMF-hosted wikis run in a realm which we can call wikis production whereas CloudVPS runs in a realm called cloud production. Network connections happening between the two realms should not have any network special treatment or exception, therefore the need to introduce the NAT. When the NAT is in place, WMF-hosted wikis will see and handle connections from CloudVPS as they would with any other internet client.
Other reason for this change is that this exceptions has been identified as requiring a lot of attention in the sense of burden to properly maintain. By removing it we are trying to reduce engineering technical debt and ease maintenance.
This change is one of the smaller pieces in a bigger architectural change that we will be introducing in upcoming months, as part of the 2020 network refresh project.
- phabricator T209011: Change routing to ensure that traffic originating from Cloud VPS is seen as non-private IPs by Wikimedia wikis
- https://gerrit.wikimedia.org/r/c/operations/puppet/+/656883 -- relevant operations/puppet.git change
- https://gerrit.wikimedia.org/r/c/operations/homer/public/+/656886 -- relevant operations/homer/public.git change