You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

SLO: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Wolfgang Kandek
imported>BCornwall
m (→‎Service Level Indicator: simple capitalization)
 
(8 intermediate revisions by 4 users not shown)
Line 2: Line 2:


Rationale: We’d love it if all our systems responded instantly and worked 100% of the time, but we also know that’s unrealistic. By choosing specific objectives based on what is important to our users, we can aim to keep our users happy, and still be able to prioritize other work as long as we’re meeting those objectives. If the performance starts to dip down toward the threshold, objectively we know it’s time to refocus on short-term reliability work and put other things on hold. And by breaking up our complex production landscape into individual services, each with its own SLOs, we know where to focus that work.
Rationale: We’d love it if all our systems responded instantly and worked 100% of the time, but we also know that’s unrealistic. By choosing specific objectives based on what is important to our users, we can aim to keep our users happy, and still be able to prioritize other work as long as we’re meeting those objectives. If the performance starts to dip down toward the threshold, objectively we know it’s time to refocus on short-term reliability work and put other things on hold. And by breaking up our complex production landscape into individual services, each with its own SLOs, we know where to focus that work.
== Service Level Indicator ==
== Service Level Indicator ==
a measurement of a behavior of a system that can be used to monitor the system's function. In the Service Level context ideally expressed as percentage
A measurement of a behavior of a system that can be used to monitor the system's function. In the Service Level context ideally expressed as percentage


=== Examples ===  
=== Examples ===  
Line 26: Line 24:
* [[SLO/etcd main cluster|etcd main cluster]]
* [[SLO/etcd main cluster|etcd main cluster]]
* [[SLO/API Gateway|API Gateway]]
* [[SLO/API Gateway|API Gateway]]
*[[SLO/Docker-registry|docker-registry]] (draft)
* [[SLO/Docker-registry|docker-registry]]
* [[SLO/Varnish|Varnish caching]]
* [[SLO/logstash|Logstash]]


== External links ==
==SLO reporting ==
Quarterly, but offset by one month, i.e. December, January, February.


* Introduction to SLOs at the Wikimedia Foundation - [https://docs.google.com/presentation/d/1XJ-FnshjHzY2NkbWEz8xGSY6GOQjO2o_u_-QsML6KyM/edit#slide=id.g18c2b41bb4_0_160 Google Slide Link]
Example:
* Implementing Service Level Objectives - [https://learning.oreilly.com/library/view/implementing-service-level/9781492076803/ O'Reilly book] - [https://docs.google.com/document/d/1rH_SxQuK5hUq3kPFmQ08EblJxOuR-z47GangoM67-FA/edit Sample Chapters]
[[File:SLOs Q3 FY20-21 Tuning Session.pdf|alt=slide of Tuning Sessions for SLOs|none|thumb|slide of Tuning Sessions for SLOs]]
* [https://sre.google/sre-book/service-level-objectives/ Service Level Objective Intro] in Site Reliability Engineering (SRE) book by Google
 
* [https://queue.acm.org/detail.cfm?id=3096459 Dependency Math] at Google
==External links==
* [https://www.usenix.org/sites/default/files/conference/protected-files/srecon18emea_slides_fong-jones.pdf SLO Workshop]
 
*Introduction to SLOs at the Wikimedia Foundation - [https://docs.google.com/presentation/d/1XJ-FnshjHzY2NkbWEz8xGSY6GOQjO2o_u_-QsML6KyM/edit#slide=id.g18c2b41bb4_0_160 Google Slide Link]
*Implementing Service Level Objectives - [https://learning.oreilly.com/library/view/implementing-service-level/9781492076803/ O'Reilly book] - [https://docs.google.com/document/d/1rH_SxQuK5hUq3kPFmQ08EblJxOuR-z47GangoM67-FA/edit Sample Chapters]
*[https://sre.google/sre-book/service-level-objectives/ Service Level Objective Intro] in Site Reliability Engineering (SRE) book by Google
*[https://queue.acm.org/detail.cfm?id=3096459 Dependency Math] at Google
*[https://blog.acolyer.org/2020/02/26/meaningful-availability/] Meaningful availability - downtime multiplied with the number of users online
*[https://www.usenix.org/sites/default/files/conference/protected-files/srecon18emea_slides_fong-jones.pdf SLO Workshop] by Google at SRECon 2018


Service Level Objectives at the foundation
Service Level Objectives at the foundation
* [https://docs.google.com/document/d/1aDPnadRRZFgGATaZ9rqjWNvvWdS6B4F50flMDVPeT1s/edit#heading=h.al2mmiq515rz Worksheet template] at SRECon 2018
*[https://docs.google.com/document/d/1aDPnadRRZFgGATaZ9rqjWNvvWdS6B4F50flMDVPeT1s/edit#heading=h.al2mmiq515rz Worksheet template]




Additional Keywords: Error Budget, [https://www.weave.works/blog/the-red-method-key-metrics-for-microservices-architecture/ RED Method], [https://sre.google/sre-book/monitoring-distributed-systems/#xref_monitoring_golden-signals four golden signals]
Additional Keywords: Error Budget, [https://www.weave.works/blog/the-red-method-key-metrics-for-microservices-architecture/ RED Method], [https://sre.google/sre-book/monitoring-distributed-systems/#xref_monitoring_golden-signals four golden signals]
[[Category:SRE Service Operations]]

Latest revision as of 15:23, 24 May 2022

Service Level Objective (SLO) and Service Level Indicators (SLI)

Rationale: We’d love it if all our systems responded instantly and worked 100% of the time, but we also know that’s unrealistic. By choosing specific objectives based on what is important to our users, we can aim to keep our users happy, and still be able to prioritize other work as long as we’re meeting those objectives. If the performance starts to dip down toward the threshold, objectively we know it’s time to refocus on short-term reliability work and put other things on hold. And by breaking up our complex production landscape into individual services, each with its own SLOs, we know where to focus that work.

Service Level Indicator

A measurement of a behavior of a system that can be used to monitor the system's function. In the Service Level context ideally expressed as percentage

Examples

  • Speed of Response: requests that are handled under a set threshold - "requests that are fulfilled under 500 ms divided by all requests * 100"
  • Success: requests that are handled error free - "all requests but return code 500 divided by all requests *100"
  • Freshness: pages that are served updated - "pages served are outdated less than 5 seconds/all pages served * 100"

Service Level Objective

Once we have SLIs we can reason about an objective

Examples

  • We want 99% of all request to be faster than 500ms
  • No more than 0.1% of errors
  • At most 1% of pages are served outdated

Published SLOs

SLO reporting

Quarterly, but offset by one month, i.e. December, January, February.

Example:

File:SLOs Q3 FY20-21 Tuning Session.pdf
slide of Tuning Sessions for SLOs

External links

Service Level Objectives at the foundation


Additional Keywords: Error Budget, RED Method, four golden signals