You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

robots.txt

From Wikitech-static
Revision as of 23:53, 12 May 2016 by imported>BryanDavis
Jump to navigation Jump to search

The rewrite rule

RewriteRule ^/robots.txt$ /w/robots.php [L]

sends all requests for robots.txt through robots.php. This script checks whether a page Mediawiki:robots.txt exists on the wiki. If it exists, the content of that page will be sent first. The file /apache/common/robots.txt will be sent afterwards.

To edit the static file, do the following:

  • Edit /home/wikipedia/common/robots.txt
  • Run sync-common-file robots.txt

The source of robots.php is available here: <https://gerrit.wikimedia.org/r/gitweb?p=operations/mediawiki-config.git;a=blob;f=live-1.5/robots.php;hb=HEAD>.