You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Help:Toolforge/Web/Lighttpd: Difference between revisions
imported>BryanDavis (→Default configuration: verified that default config is up to date by comparing to running container) |
imported>BryanDavis (Use <syntaxhighlight lang="lighttpd"> a lot) |
||
| Line 24: | Line 24: | ||
This is the default (if you don't specify any other/additional settings in your tool's .lighttpd.conf) | This is the default (if you don't specify any other/additional settings in your tool's .lighttpd.conf) | ||
{{Collapse top|Default lighttpd configuration}} | {{Collapse top|Default lighttpd configuration}} | ||
< | <syntaxhighlight lang="lighttpd"> | ||
server.modules = ( | server.modules = ( | ||
"mod_setenv", | "mod_setenv", | ||
| Line 87: | Line 87: | ||
)) | )) | ||
) | ) | ||
</ | </syntaxhighlight> | ||
(config as of 2018-12-05) | (config as of 2018-12-05) | ||
{{Collapse bottom}} | {{Collapse bottom}} | ||
| Line 96: | Line 96: | ||
=== FCGI Flask config === | === FCGI Flask config === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
fastcgi.server += ( "/gerrit-patch-uploader" => | fastcgi.server += ( "/gerrit-patch-uploader" => | ||
(( | (( | ||
| Line 105: | Line 105: | ||
)) | )) | ||
) | ) | ||
</ | </syntaxhighlight> | ||
For Flask, the fcgi handler looks like this: https://github.com/valhallasw/gerrit-patch-uploader/blob/master/app.fcgi | For Flask, the fcgi handler looks like this: https://github.com/valhallasw/gerrit-patch-uploader/blob/master/app.fcgi | ||
| Line 114: | Line 114: | ||
Note that rewrite rules always execute before redirect rules (regardless of their order in the config file). | Note that rewrite rules always execute before redirect rules (regardless of their order in the config file). | ||
< | A common rewrite scenario is a PHP application that uses a [[w:Front controller|front controller]] script (often ''index.php'') to decide how to process each request based on the path of the request. The '''url.rewrite-if-not-file''' directive can be used to do this while still letting lighttpd serve static files from ''$HOME/public_html'' normally: | ||
url.rewrite- | <syntaxhighlight lang="lighttpd"> | ||
url.rewrite-if-not-file += ( "(.*)" => "/YOUR_TOOL_NAME/index.php/$0" ) | |||
</ | </syntaxhighlight> | ||
Alternately you can use '''url.rewrite-once''' to pick and choose which requests to rewrite: | |||
<syntaxhighlight lang="lighttpd"> | |||
< | url.rewrite-once += ( | ||
url.rewrite-once = ( | |||
".*\.(js|css)" => "$0", | ".*\.(js|css)" => "$0", | ||
"^/ | "^/YOUR_TOOL_NAME(/.*)" => "/YOUR_TOOL_NAME/index.php$1" | ||
) | ) | ||
</ | </syntaxhighlight> | ||
<syntaxhighlight lang="lighttpd"> | |||
url.rewrite-once += ( "/YOUR_TOOL_NAME/id/([0-9]+)" => "/YOUR_TOOL_NAME/index.php?id=$1", | |||
"/YOUR_TOOL_NAME/link/([a-zA-Z]+)" => "/YOUR_TOOL_NAME/index.php?link=$1" ) | |||
</syntaxhighlight> | |||
The | The <code>$0</code> matches the entire match from the left-hand side regular expression. | ||
=== Header, mimetype, character encoding, error handler === | === Header, mimetype, character encoding, error handler === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# Allow Cross-Origin Resource Sharing (CORS) | # Allow Cross-Origin Resource Sharing (CORS) | ||
setenv.add-response-header += ( "Access-Control-Allow-Origin" => "en.wikipedia.org", | setenv.add-response-header += ( "Access-Control-Allow-Origin" => "en.wikipedia.org", | ||
| Line 152: | Line 156: | ||
# Add custom error-404 handler | # Add custom error-404 handler | ||
server.error-handler-404 += "/error-404.php" | server.error-handler-404 += "/error-404.php" | ||
</ | </syntaxhighlight> | ||
Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModSetEnv ModSetEnv] [//redmine.lighttpd.net/projects/lighttpd/wiki/Mimetype_assignDetails Mimetype-Assign] [//redmine.lighttpd.net/projects/lighttpd/wiki/Server_error-handler-404Details Error-Handler-404] | Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModSetEnv ModSetEnv] [//redmine.lighttpd.net/projects/lighttpd/wiki/Mimetype_assignDetails Mimetype-Assign] [//redmine.lighttpd.net/projects/lighttpd/wiki/Server_error-handler-404Details Error-Handler-404] | ||
| Line 158: | Line 162: | ||
=== {{anchor|lighttpd-conf-index}}Directory or file index === | === {{anchor|lighttpd-conf-index}}Directory or file index === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# Enable basic directory index | # Enable basic directory index | ||
$HTTP["url"] =~ "^/?" { | $HTTP["url"] =~ "^/?" { | ||
dir-listing.activate = "enable" | dir-listing.activate = "enable" | ||
} | } | ||
</ | </syntaxhighlight> | ||
=== {{anchor|lighttpd-conf-hidden}}Deny access to hidden files === | === {{anchor|lighttpd-conf-hidden}}Deny access to hidden files === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# Deny access to hidden files | # Deny access to hidden files | ||
$HTTP["url"] =~ "/\." { | $HTTP["url"] =~ "/\." { | ||
url.access-deny = ("") | url.access-deny = ("") | ||
} | } | ||
</ | </syntaxhighlight> | ||
Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModAccess ModAccess] | Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModAccess ModAccess] | ||
=== Custom index === | === Custom index === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# Enable index for specific directory | # Enable index for specific directory | ||
$HTTP["url"] =~ "^/download($|/)" { | $HTTP["url"] =~ "^/download($|/)" { | ||
| Line 183: | Line 187: | ||
# Custom index file or custom directory generator | # Custom index file or custom directory generator | ||
index-file.names += ("index.py") | index-file.names += ("index.py") | ||
</ | </syntaxhighlight> | ||
Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModDirlisting ModDirlisting] | Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModDirlisting ModDirlisting] | ||
| Line 191: | Line 195: | ||
Add the line: | Add the line: | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# Enable request logging | # Enable request logging | ||
debug.log-request-handling = "enable" | debug.log-request-handling = "enable" | ||
</ | </syntaxhighlight> | ||
The debug output will be written to the <code>error.log</code> file. | The debug output will be written to the <code>error.log</code> file. | ||
=== Apache-like cgi-bin directory === | === Apache-like cgi-bin directory === | ||
Add the following stanza: | Add the following stanza: | ||
< | <syntaxhighlight lang="lighttpd"> | ||
$HTTP["url"] =~ "^/your_tool/cgi-bin" { | $HTTP["url"] =~ "^/your_tool/cgi-bin" { | ||
cgi.assign = ( "" => "" ) | cgi.assign = ( "" => "" ) | ||
} | } | ||
</ | </syntaxhighlight> | ||
This does require that cgi-bin be ''under'' your public_html rather than alongside it. | This does require that cgi-bin be ''under'' your public_html rather than alongside it. | ||
| Line 209: | Line 213: | ||
To run CGI from any directory under your public_html only need this one line (w/out the $HTTP["url"] .. block) | To run CGI from any directory under your public_html only need this one line (w/out the $HTTP["url"] .. block) | ||
< | <syntaxhighlight lang="lighttpd"> | ||
cgi.assign += ( ".cgi" => "" ) | cgi.assign += ( ".cgi" => "" ) | ||
</ | </syntaxhighlight> | ||
The part to the left is the file name or extension ("" = any). The part to the right is the program which will run it ("" = any). Another example | The part to the left is the file name or extension ("" = any). The part to the right is the program which will run it ("" = any). Another example | ||
< | <syntaxhighlight lang="lighttpd"> | ||
cgi.assign += ( "script.sh" => "/bin/bash" ) | cgi.assign += ( "script.sh" => "/bin/bash" ) | ||
</ | </syntaxhighlight> | ||
=== Enable status and statistics === | === Enable status and statistics === | ||
< | <syntaxhighlight lang="lighttpd"> | ||
# modify <toolname> for your tool | # modify <toolname> for your tool | ||
# this will enable counters http://tools.wmflabs.org/<toolname>/server-status (resp: .../server-statistics) | # this will enable counters http://tools.wmflabs.org/<toolname>/server-status (resp: .../server-statistics) | ||
| Line 226: | Line 230: | ||
status.status-url = "/<toolname>/server-status" | status.status-url = "/<toolname>/server-status" | ||
status.statistics-url = "/<toolname>/server-statistics" | status.statistics-url = "/<toolname>/server-statistics" | ||
</ | </syntaxhighlight> | ||
Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModStatus ModStatus] | Details: [//redmine.lighttpd.net/projects/1/wiki/Docs_ModStatus ModStatus] | ||
Revision as of 20:53, 14 December 2018
Lighttpd is the HTTP server used by both the lighttpd and lighttpd-plain types supported by webservice. These types are supported by both the Grid Engine and Kubernetes backends.
- The document root is
$HOME/public_html.- See Changing the document root for some ideas on how to change that.
- Error logs from the lighttpd process are stored in
$HOME/error.log - PHP scripts are automatically run using a FastCGI helper process.
- The lighttpd web server is configurable (including adding other FastCGI handlers). A
$HOME/.lighttpd.conffile can be used to change the default configuration. - Everything runs as the tool user, regardless of file ownership.
The web server reads any configuration in $HOME/.lighttpd.conf, and merges it with the default configuration. Most tools will not need custom configuration.
Sometimes merge fails if an option is already set in the default configuration. When this happens, try using option += value instead of option = value.
Default configuration
This is the default (if you don't specify any other/additional settings in your tool's .lighttpd.conf)
| Default lighttpd configuration |
|---|
| The following content has been placed in a collapsed box for improved usability. |
server.modules = (
"mod_setenv",
"mod_access",
"mod_accesslog",
"mod_alias",
"mod_compress",
"mod_redirect",
"mod_rewrite",
"mod_fastcgi",
"mod_cgi",
)
server.port = {port}
server.use-ipv6 = "disable"
server.username = "{username}"
server.groupname = "{groupname}"
server.core-files = "disable"
server.document-root = "{home}/public_html"
server.pid-file = "/var/run/lighttpd/{toolname}.pid"
server.errorlog = "{home}/error.log"
server.breakagelog = "{home}/error.log"
server.follow-symlink = "enable"
server.max-connections = 300
server.stat-cache-engine = "simple"
server.event-handler = "linux-sysepoll"
ssl.engine = "disable"
alias.url = ( "/{toolname}" => "{home}/public_html/" )
index-file.names = ( "index.php", "index.html", "index.htm" )
dir-listing.encoding = "utf-8"
server.dir-listing = "disable"
url.access-deny = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
accesslog.use-syslog = "disable"
accesslog.filename = "{home}/access.log"
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
cgi.assign = (
".pl" => "/usr/bin/perl",
".py" => "/usr/bin/python",
".pyc" => "/usr/bin/python",
)
fastcgi.server += ( ".php" =>
((
"bin-path" => "/usr/bin/php-cgi",
"socket" => "/var/run/lighttpd/php.socket.{toolname}",
"max-procs" => 2,
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "2",
"PHP_FCGI_MAX_REQUESTS" => "500"
),
"bin-copy-environment" => (
"PATH", "SHELL", "USER"
),
"broken-scriptfilename" => "enable",
"allow-x-send-file" => "enable"
))
)
(config as of 2018-12-05) |
| The above content has been placed in a collapsed box for improved usability. |
See lighttpdwebservice.py in operations/software/tools/webservice for the canonical configuration.
Example configurations
FCGI Flask config
fastcgi.server += ( "/gerrit-patch-uploader" =>
((
"socket" => "/tmp/patchuploader-fcgi.sock",
"bin-path" => "/data/project/gerrit-patch-uploader/src/gerrit-patch-uploader/app.fcgi",
"check-local" => "disable",
"max-procs" => 1,
))
)
For Flask, the fcgi handler looks like this: https://github.com/valhallasw/gerrit-patch-uploader/blob/master/app.fcgi
URL rewrite
- Documentation: ModRewrite
Note that rewrite rules always execute before redirect rules (regardless of their order in the config file).
A common rewrite scenario is a PHP application that uses a front controller script (often index.php) to decide how to process each request based on the path of the request. The url.rewrite-if-not-file directive can be used to do this while still letting lighttpd serve static files from $HOME/public_html normally:
url.rewrite-if-not-file += ( "(.*)" => "/YOUR_TOOL_NAME/index.php/$0" )
Alternately you can use url.rewrite-once to pick and choose which requests to rewrite:
url.rewrite-once += (
".*\.(js|css)" => "$0",
"^/YOUR_TOOL_NAME(/.*)" => "/YOUR_TOOL_NAME/index.php$1"
)
url.rewrite-once += ( "/YOUR_TOOL_NAME/id/([0-9]+)" => "/YOUR_TOOL_NAME/index.php?id=$1",
"/YOUR_TOOL_NAME/link/([a-zA-Z]+)" => "/YOUR_TOOL_NAME/index.php?link=$1" )
The $0 matches the entire match from the left-hand side regular expression.
Header, mimetype, character encoding, error handler
# Allow Cross-Origin Resource Sharing (CORS)
setenv.add-response-header += ( "Access-Control-Allow-Origin" => "en.wikipedia.org",
"Access-Control-Allow-Methods" => "POST, GET, OPTIONS" )
# Set cache-control directive for static files and resources
$HTTP["url"] =~ "\.(jpg|gif|png|css|js|txt|ico)$" {
setenv.add-response-header += ( "Cache-Control" => "max-age=86400, public" )
}
mimetype.assign += (
# Add custom mimetype
".bulk" => "text/plain",
# Avoid [[Mojibake]] in JavaScript files
".js" => "application/javascript; charset=utf-8",
# Default MIME type with UTF-8 character encoding
"" => "text/plain; charset=utf-8"
)
# Add custom error-404 handler
server.error-handler-404 += "/error-404.php"
Details: ModSetEnv Mimetype-Assign Error-Handler-404 HTTP access control (CORS)
Directory or file index
# Enable basic directory index
$HTTP["url"] =~ "^/?" {
dir-listing.activate = "enable"
}
# Deny access to hidden files
$HTTP["url"] =~ "/\." {
url.access-deny = ("")
}
Details: ModAccess
Custom index
# Enable index for specific directory
$HTTP["url"] =~ "^/download($|/)" {
dir-listing.activate = "enable"
}
# Custom index file or custom directory generator
index-file.names += ("index.py")
Details: ModDirlisting
Request logging
- Documentation: DebugVariables
Add the line:
# Enable request logging
debug.log-request-handling = "enable"
The debug output will be written to the error.log file.
Apache-like cgi-bin directory
Add the following stanza:
$HTTP["url"] =~ "^/your_tool/cgi-bin" {
cgi.assign = ( "" => "" )
}
This does require that cgi-bin be under your public_html rather than alongside it.
To run CGI from any directory under your public_html only need this one line (w/out the $HTTP["url"] .. block)
cgi.assign += ( ".cgi" => "" )
The part to the left is the file name or extension ("" = any). The part to the right is the program which will run it ("" = any). Another example
cgi.assign += ( "script.sh" => "/bin/bash" )
Enable status and statistics
# modify <toolname> for your tool
# this will enable counters http://tools.wmflabs.org/<toolname>/server-status (resp: .../server-statistics)
server.modules += ("mod_status")
status.status-url = "/<toolname>/server-status"
status.statistics-url = "/<toolname>/server-statistics"
Details: ModStatus
Web logs
Your tool's web logs are placed in the tool account's $HOME/access.log in common format. Please note that the web logs are anonymized in accordance with the Foundation’s privacy policy. Each user IP address will appear to be that of the local host, for example. In general, the privacy policy precludes the logging of personally identifiable information; special permission from Foundation legal counsel is required if such information is required.
Error logs can be found in the tool account's $HOME/error.log; this includes the standard error of invoked scripts.
Error pages
The proxy provides its own error pages when your application returns HTTP/500, HTTP/502 or HTTP/503. This behavior is currently under review, and might change in the near future.
You can bypass the proxy error pages by passing an X-Wikimedia-Debug header.
Changing the document root
With symlinks
The easiest way to change the document root is with a symlink to $HOME/public_html. However, before this is done, the existing public_html directory needs to be deleted or moved. This is because ln -s $HOME/foo $HOME/public_html would make $HOME/public_html/foo if the $HOME/public_html directory exists. Deleting the directory is done with rm -rf $HOME/public_html to delete the directory and all of its contents, if you do not need anything in there, or with mv $HOME/public_html $HOME/oldpublic_html to move the directory to $HOME/oldpublic_html.
To make the symlink, ln -s $HOME/foo $HOME/public_html/ would make the contents of $HOME/foo available in $HOME/public_html. Replace $HOME/foo in the example with the directory you want lighttpd to serve.
With aliases
Note that you cannot add an alias URL for /toolname because this has already been defined and can't be overridden in the local conf file.
PHP
The lighttpd webservice type includes support for running PHP scripts from files with a .php in $HOME/public_html using a FastCGI helper process.
PHP ini settings can be changed for your tool by creating a $HOME/public_html/.user.ini configuration file. This can be useful for setting the default timezone with date.timezone. See the documentation at php.net for more details.