SLOs and Error Budgets
An availability target is only useful if missing it *changes what the team does* — otherwise it is a number in a slide deck, and everyone already knows it.
What is it?#
Three terms, in increasing order of consequence:
| Is a | Missing it costs you | |
|---|---|---|
| SLI | measurement — good events ÷ valid events | Nothing. It is just a number |
| SLO | internal target for that SLI | An engineering decision you agreed in advance |
| SLA | external contract with a customer | Money |
An error budget is the arithmetic remainder: 1 − SLO, expressed as the amount of failure you are allowed. At 99.9% monthly, that is about 43 minutes.
Why it exists#
Not to measure reliability. To settle an argument before it happens.
Every team has the same recurring fight: the product side wants to ship, the operations side wants to be careful, and each round is re-argued from scratch with no shared basis for deciding.
An error budget replaces opinion with arithmetic:
| Without an error budget | With one |
|---|---|
| "Is it safe to deploy?" — reargued weekly | Check the budget |
| Reliability work competes with features | Spending the budget creates the mandate |
| The careful engineer is "blocking" | The policy blocks, not a person |
| 100% is implicitly the target | The target is explicit and affordable |
What it is made of#
The nines, in time you can actually picture#
| SLO | Downtime / month | Downtime / year | Feels like |
|---|---|---|---|
| 99% | 7.3 hours | 3.65 days | Noticeable weekly |
| 99.9% | 43.8 minutes | 8.8 hours | One bad deploy |
| 99.95% | 21.9 minutes | 4.4 hours | Tight for a small team |
| 99.99% | 4.4 minutes | 52.6 minutes | Needs multi-AZ and automation |
| 99.999% | 26 seconds | 5.3 minutes | No human can respond in time |
Choosing an SLI that means something#
The SLI has to be measured where the user is, and count what the user notices.
| Weak SLI | Why it lies | Better |
|---|---|---|
| Server CPU < 80% | Users cannot feel CPU | Request success rate at the load balancer |
| Average latency | Averages erase the slow tail | p99 latency |
| Uptime of the VM | The VM is up, the app returns 500 | Successful requests ÷ valid requests |
| Ping succeeds | Proves routing, not service | An actual request to a real endpoint |
How to use it#
Define one, concretely#
SLI: proportion of HTTP requests to /api/* returning < 500,
measured at the load balancer
SLO: 99.9% over a rolling 28 days
Budget: 0.1% = ~43 minutes of failed requests per 28 days
Policy: > 50% budget spent → no risky deploys, reliability work prioritised
> 100% spent → feature freeze until back within budgetThe policy line is the one that matters. An SLO with no policy attached changes nothing, and is the most common way this whole practice fails.
Burn rate: how to alert without being paged for nothing#
Alerting on "the SLO was missed" is useless — by then it already happened. Alert on the rate you are consuming budget.
Burn rate 1 = you will spend exactly the whole budget by the end of the window. Burn rate 10 = you will spend it in a tenth of the time.
| Burn rate | Budget gone in | Response |
|---|---|---|
| 1 | 28 days | Nothing. This is the design |
| 2 | 14 days | A ticket |
| 6 | ~5 days | Investigate today |
| 14 | 2 days | Page someone |
What goes wrong#
| Mistake | Why it fails | Instead |
|---|---|---|
| An SLO with no policy | Nothing changes when you miss it | Write the "then what" before the number |
| Setting the SLO at 100% | Removes the ability to deploy, patch or migrate | Pick a number you can afford to miss |
| Measuring on the server | The user is not on your server | Measure at the load balancer, or from the client |
| Averages instead of percentiles | The average hides the users who are leaving | p99, and watch p99.9 |
| SLO = SLA | You lose the safety margin | SLO tighter than the SLA, always |
| Too many SLOs | Twenty targets are zero targets | Two or three per user-facing journey |
| Alerting on budget exhausted | It is already too late | Alert on burn rate, on two windows |
Interview corner · الأسئلة اللي بتتسأل#
What is the difference between an SLI, an SLO and an SLA? · إيه الفرق بينهم؟
Measurement, target, contract — in increasing order of consequence.
- SLI is a number: good events ÷ valid events. Missing it costs nothing because it is not a target.
- SLO is your internal target for that number. Missing it triggers an engineering policy you agreed in advance.
- SLA is a contract with a customer, with financial penalties.
The relationship that matters: your SLO must be tighter than your SLA, so you notice and react before the customer's lawyer does. If SLO = SLA you have no margin, and the first breach is a commercial event rather than an engineering one.
Your product manager asks for 100% uptime. What do you say? · المدير عايز ١٠٠٪ إتاحة
Do not agree, and do not simply refuse. Reframe with numbers.
- 100% is not purchasable. Your cloud provider will not sign it, your upstream DNS does not have it, and the marginal cost of each nine rises roughly tenfold while the marginal user benefit falls.
- Ask what the user actually notices. Availability of what, measured where? Checkout at the load balancer is a different number from "the whole site from every country".
- Offer a real target with its price. 99.9% is 43 minutes a month and affordable. 99.99% is 4.4 minutes and needs multi-AZ, automated failover and someone on call.
- Point out the budget is a feature. With zero allowed downtime you can never deploy, patch a CVE, or migrate a database — so a 100% target makes the service less safe over time, not more.
How would you alert on an SLO without waking people up for nothing?
Alert on burn rate, not on the SLO being missed, and use two windows together.
Missing the SLO is a lagging indicator — by the time it fires, the month is already spent. Burn rate is leading: it says how fast the budget is going.
| Alert on | Result |
|---|---|
| Short window only | Pages on every transient blip; people stop reading alerts |
| Long window only | Sleeps through a real outage for an hour |
| Both, ANDed | Fast and trustworthy |
Then tier by severity: a high burn rate pages, a moderate one opens a ticket, and burn rate 1 does nothing at all — because burn rate 1 is exactly what the budget was designed to absorb.
Key takeaways#
- SLI measures, SLO targets, SLA costs money. And your SLO must be tighter than your SLA.
- An SLO with no policy attached changes nothing. Write the "then what" before you argue about the number.
- The error budget exists to settle an argument, not to measure — it makes reliability work a mandate instead of a preference.
- Measure where the user is, and use percentiles, not averages.
- Set the first SLO slightly below what you already achieve. One you always miss teaches the team to ignore it.
- Alert on burn rate over two windows, never on the budget being exhausted.
- 100% is a category error. With no allowed downtime you can never deploy or patch, which makes the service less safe.
Last updated 2026-09-08