You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

News/Toolforge.org

From Wikitech-static
< News
Revision as of 00:55, 2 April 2020 by imported>DSquirrelGM (rewrite)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

The Toolforge service will start using the toolforge.org domain, and a new URL scheme for webservices. This page describes the changes, how to handle it, and what to expect. This change is expected to primarily affect Toolforge webservices, with very few expected changes for tools that do not use a webservice interface.

Context

All webservices running on Toolforge has been traditionally served using this URL scheme:

https://tools.wmflabs.org/<toolname>

This means that all webservices running in Toolforge share the same domain (tools.wmflabs.org), even though they are different tools, managed by different developers, with different source code, different purposes, and different scopes.

For years now, we have been trying to provide a better URL/domain isolation for webservices. We want each webservice to have its own domain. Also, we wanted to introduce a more proper domain for the Toolforge service, toolforge.org.

The new domain also helps us get a step closer in moving away from the labs keyword. Toolforge is a established service for the Wikimedia community and should not be considered labs anymore.

For all the reasons above, we have been trying really hard to see how and when is best to introduce the new domain. The moment is upon us now :-)

New URL for webservices

Webservices running in Toolforge will be served now in a new URL scheme:

  • from:
    https://tools.wmflabs.org/<toolname>
  • to:
    https://<toolname>.toolforge.org/

Please note the document root has changed. Your webservices previosly got /<toolname>/ as the document root and now is just /.

What should I do?

There are a couple of things you can do to make sure your tool webservice is ready for using the new domain toolforge.org and the new URL.

Here is a checklist that you can follow:

  • make sure your SSH configuration points to login.toolforge.org.
  • make sure your webservice source code understand and is ready to work with the new domain, i.e. from tools.wmflabs.org to toolforge.org.
  • make sure your webservice source code understand and is ready to work with the new URL document root, i.e. from /<toolname>/ to /.
  • try starting your webservice with the --canonical argument and evaluate how your webservice behaves with this redirection.
  • some webservices running in the new Toolforge kubernetes cluster may need to restart the webservice, so the new ingress object is created.

See the Adopting the change section below for details.

Adopting the change

We considered having a compatibility period in which tool developers can adapt their webservices to the new domain and URL scheme. During this period, you should test and adapt your webservice as required to make sure it works with the new environment.

the --canonical switch

During the compatibility period, the webservice utility has a command line switch to introduce a redirect from the legacy URL scheme to the new one:

  • webservice --canonical --backend=gridengine start [..]
  • webservice --canonical --backend=kubernetes start [..]

When you start your webservice like this a mechanism will be activated to automatically redirect incoming requests by using the HTTP status code 307 Temporary Redirect. Example of such redirection:

  • from:
    https://tools.wmflabs.org/mytool/index.html?param1=true&param2=false
  • to:
    https://mytool.toolforge.org/index.html?param1=true&param2=false

Note how the tool name was removed from the path, but URL parameters and arguments are kept the same.
When the redirect mechanism is activated, requests for the legacy URL wont reach your webservice, as the redirection happens on an external component. Your webservice will only get requests in the new domain and URL scheme.

If you stop your webservice and start it again without the --canonical argument, everything will get back to normal legacy behavior and URL scheme, with no redirection.

This allows for on-demand testing and development associated with the URL scheme change.
When the compatibility period is over, the --canonical command line switch will no longer be available (see Timeline and After migration period sections below).

interwiki links

TODO: talk about this, if it is important here anyway: iw tool.

Oauth considerations

TODO: write some info.

Language/runtime specific hints

Find here some specific adaptation hints for different langs and web frameworks.

django

TODO: any info to put here?

flask

TODO: any info to put here?

Under the legacy scheme, it was necessary to configure the APPLICATION_ROOT (or directly the SESSION_COOKIE_PATH) to ensure that the session cookie would only be sent to your tool and not other tools on tools.wmflabs.org. Under the new scheme, this is no longer necessary; if you followed the instructions at Help:Toolforge/My first Flask OAuth tool, the fix is to remove the APPLICATION_ROOT line from the tool’s ~/www/python/src/config.yaml file:

# caution: this command has not yet been tested (TODO)
sed -i.bak '/APPLICATION_ROOT/d' ~/www/python/src/config.yaml

vanilla php

TODO: any info to put here?

vanilla python

TODO: any info to put here?

SSH changes

There are a new FQDNs for the bastion SSH services.

  • old/legacy: login.tools.wmflabs.org
  • new: login.toolforge.org

And:

  • old/legacy: dev.tools.wmflabs.org
  • new: dev.toolforge.org

Please make sure your SSH configuration is up-to-date with the FQDNs. The SSH host key fingerprints are the same and may still be found at Help:SSH Fingerprints/login.tools.wmflabs.org and Help:SSH Fingerprints/stretch-dev.tools.wmflabs.org, respectively.

After migration period

Toolforge legacy redirect.png

When the migration period is over, we expect every webservice running in Toolforge to be working well with the new domain and URL scheme.

We plan to preserve old URLs by introducing a legacy redirector which will work mostly like what the --canonical option does.

Every request to URLs in the legacy scheme will be receive a HTTP status code 301 moved permanently.

The list of tools that will receive this redirection will be static, and when the migration period is over new webservices will only be allowed in the new scheme.

Timeline

This is a timeline of events related to this change.

  • 2020-04-06: initial announce to the community. Compatibility period starts.
  • 2020-04-30: evaluate compatibility adoption. Toolforge users/developers reach out for help if required.
  • 2020-05-31: Soft deadline. Evaluate compatibility adoption & help webservices/users/developers struggling with this change. Start deploying the legacy redirector system.
  • 2020-06-15: Hard deadline. Compatibility period is over and all HTTP request to the legacy URLs are redirected to the new domain. The old tools.wmflabs.org domain is no longer allowed for new tools, which now get toolforge.org by default.

See also

Some other information that might be relevant related to this topic: