You are browsing a read-only backup copy of Wikitech. The live site can be found at

Robots.txt: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
(updated page)
(BCornwall moved page Robots.txt to Obsolete:Robots.txt: Ancient knowledge that wasn't corrected for over 7 years, so it's likely not useful knowledge. If it ever is needed it belongs as part of a larger article)
(One intermediate revision by one other user not shown)
Line 1: Line 1:
{{lowercase title}}
#REDIRECT [[Obsolete:Robots.txt]]
The rewrite rule
RewriteRule ^/robots.txt$ /w/robots.php [L]
sends all requests for robots.txt through robots.php. This script checks whether a page ''Mediawiki:robots.txt'' exists on the wiki. If it exists, the content of that page will be sent first. The file /apache/common/robots.txt will be sent afterwards.
To edit the static file, do the following:
* Edit <tt>/home/wikipedia/common/robots.txt</tt>
* Run  <tt>sync-common-file robots.txt</tt>
The source of robots.php is available here: <;a=blob;f=live-1.5/robots.php;hb=HEAD>.

Latest revision as of 19:33, 2 June 2022