You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Runbook: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Krinkle
No edit summary
imported>Andrew Bogott
No edit summary
 
(5 intermediate revisions by 4 users not shown)
Line 1: Line 1:
A runbook is a set of instructions for a human what to do. More specifically what to do when a certain [[monitoring]] alert triggers.
A runbook is a set of instructions for a human what to do. More specifically what to do when a certain [[monitoring]] alert triggers.


Pages that contain runbooks are linked from [[Icinga]] checks in [[puppet]] using the *notes_url* parameter.
Prometheus alerting rules link to this page in a service-specific anchor based on the service name itself; from the service entry you can give more context, next actions, and link to other resources such as more runbooks, service dashboards, etc.
 
Pages that contain runbooks are linked from [[Icinga]] checks in [[puppet]] using the <tt>notes_url</tt> parameter, or alertmanager rules with the <tt>runbook</tt> annotation.  


There is a [[:Category:Runbooks]] for pages with runbooks.
There is a [[:Category:Runbooks]] for pages with runbooks.


Compare to [[cookbooks]] which are programs running in [[spicerack]]/[[cumin]] to do maintenance tasks.
Compare to [[cookbooks]] which are programs running in [[spicerack]]/[[cumin]] to do maintenance tasks.
== service-name:port ==
This is the entry linked from generic alerts for <tt>service-name:port</tt>, for example <tt>ProbeDown</tt> will link here when network probes for <tt>service-name</tt> fail. See also {{bug|T312947}}.


[[Category:Runbooks| ]]
[[Category:Runbooks| ]]
[[Category:Monitoring]]
[[Category:Monitoring]]
== apt:80 ==
See [[APT_repository]]
== grafana:443 ==
See [[Grafana.wikimedia.org]]
== graphite:443 ==
See [[Graphite]]
== helm-charts:443 ==
This is the service powered by chartmuseum, see [[ChartMuseum]]
== jobrunner:443 ==
See [[Application servers/Runbook#Jobrunners]]
== librenms:443 ==
See [[LibreNMS]]
== netbox:443 ==
See [[Netbox]]
== puppetboard:443 ==
See [[Puppet]]
== puppetdb-api:8090 ==
See [[Puppet#Micro_Service]]
== releases:443 ==
See [[Releases.wikimedia.org]]
== upload-https:443 ==
Check [https://logstash.wikimedia.org/app/dashboards#/view/ee6432c0-82a9-11eb-9d45-739221ba7fb6 NEL]
== videoscaler:443 ==
See [[Application servers/Runbook#Jobrunners]]
== tools-k8s-haproxy-3:30000 ==
See [[Portal:Toolforge/Admin/Runbooks/k8s-haproxy]]
== tools-k8s-haproxy-4:30000 ==
See [[Portal:Toolforge/Admin/Runbooks/k8s-haproxy]]

Latest revision as of 16:22, 2 September 2022

A runbook is a set of instructions for a human what to do. More specifically what to do when a certain monitoring alert triggers.

Prometheus alerting rules link to this page in a service-specific anchor based on the service name itself; from the service entry you can give more context, next actions, and link to other resources such as more runbooks, service dashboards, etc.

Pages that contain runbooks are linked from Icinga checks in puppet using the notes_url parameter, or alertmanager rules with the runbook annotation.

There is a Category:Runbooks for pages with runbooks.

Compare to cookbooks which are programs running in spicerack/cumin to do maintenance tasks.

service-name:port

This is the entry linked from generic alerts for service-name:port, for example ProbeDown will link here when network probes for service-name fail. See also bug T312947.

apt:80

See APT_repository

grafana:443

See Grafana.wikimedia.org

graphite:443

See Graphite

helm-charts:443

This is the service powered by chartmuseum, see ChartMuseum

jobrunner:443

See Application servers/Runbook#Jobrunners

librenms:443

See LibreNMS

netbox:443

See Netbox

puppetboard:443

See Puppet

puppetdb-api:8090

See Puppet#Micro_Service

releases:443

See Releases.wikimedia.org

upload-https:443

Check NEL

videoscaler:443

See Application servers/Runbook#Jobrunners

tools-k8s-haproxy-3:30000

See Portal:Toolforge/Admin/Runbooks/k8s-haproxy

tools-k8s-haproxy-4:30000

See Portal:Toolforge/Admin/Runbooks/k8s-haproxy