slo-architect

---
name: slo-architect
description: Use when defining or reviewing SLOs/SLIs/error budgets, sizing burn-rate alerts, or auditing SLO docs for missing SLI definitions, bad windows, or no error-budget policy.
category: Engineering
version: 1.0.0
tools: []
---

# SLO Architect

Define SLOs that mean something. Most "SLOs" in the wild are arbitrary numbers no one believes — 99.9% on every endpoint, no SLI definition, no error budget, no policy for what happens when budget burns. Enforce the discipline from Google's SRE Workbook: pick the right SLI, set a target users actually care about, calculate the error budget, reason about multi-window burn-rate alerts, and have a written policy for when budget runs out.

## When to use

- Defining a new SLO for a service or feature
- Reviewing existing SLOs for common bugs
- Picking the right SLI (event-based vs time-window based vs request-based)
- Computing error budgets and burn-rate alert thresholds
- Tying SLOs to existing controls — feature-flag rollout aborts, chaos-engineering blast radius, operator capability levels

## When NOT to use

- General observability strategy (metrics + logs + traces) — that's a broader observability discipline, not SLO-specific
- Customer-facing SLAs with legal teeth — that's contract drafting, not engineering
- Performance load testing (capacity, not reliability)
- Active incident response

## Core principle: an SLO is a promise about user experience

```
SLI  -> measurable signal of user-perceived health (e.g., HTTP 2xx rate)
SLO  -> target for the SLI over a window (e.g., 99.9% over 30 days)
SLA  -> customer-facing commitment with consequences (separate concern)
EB   -> error budget: 100% minus SLO target = how much "bad" you can spend
BR   -> burn rate: how fast you're consuming the error budget
```

The four cardinal mistakes:

1. **Target too high** (99.99%+ on services that can't support it) — every minor blip violates SLO; alerts become noise.
2. **Wrong SLI** (CPU usage as proxy for user experience) — system can be "green" while users suffer.
3. **No error budget policy** — burning budget means nothing if there's no agreed action.
4. **Single-window burn-rate alert** — either too noisy (page on a 5-min spike) or too slow (notice budget exhausted after the fact).

## SLI selection cheatsheet

| User experience | SLI type | What you measure |
|---|---|---|
| "Did the request succeed?" | request-success-rate | `2xx / total` |
| "Was the response fast?" | request-latency | `count(p99 < threshold) / total` |
| "Was the service up?" | availability-time | `(window - downtime) / window` |
| "Is the data current?" | data-freshness | `count(data_age < threshold) / total` |
| "Was the answer correct?" | correctness | `count(correct) / total` |

## SLO definition checklist

Every SLO needs: service name, SLI type and precise numerator/denominator, target, window, owner. Refuse to call an SLO "defined" if any of these is missing — mark it `<must define>` and keep pushing.

When reviewing existing SLO definitions (in chat, from pasted docs, or via knowledge_base_search), check for these common bugs:

- `target_too_high`: target >= 99.99% (sustainable only with massive engineering investment)
- `target_too_low`: target <= 99.0% (probably wrong SLI; users will notice)
- `window_too_short`: window < 7 days (statistical noise dominates)
- `window_too_long`: window > 90 days (slow feedback)
- `no_sli_definition`: SLI section missing or vague ("everything OK")
- `no_error_budget_policy`: no documented action when budget burns
- `cpu_as_sli`: CPU/memory used as user-experience proxy (wrong signal)

## Error budget math

For a 99.9% SLO over 30 days:
- Allowed unavailability: `0.1% x 30 x 24 x 60 = 43.2 minutes`
- Multi-window burn-rate alerts, per Google SRE Workbook Chapter 5:
  - **Fast burn** — page if 2% of the monthly budget is consumed in 1 hour (~1.44x ratio multiplier on the 1-hour threshold)
  - **Slow burn** — page if 10% consumed in 6 hours (~0.6x ratio multiplier), open a ticket if 10% in 3 days

Work these numbers by hand or reason through them explicitly when advising a team — don't just assert a threshold, show the arithmetic so it's auditable.

## Composition with other reliability disciplines

An SLO isn't just a target — it should feed the mechanisms that act on it:

| Concern | Composition |
|---|---|
| Feature-flag rollouts | Rollout abort criteria should reference SLO burn-rate thresholds |
| Chaos engineering | Blast-radius calculations should take the monthly error budget as an input |
| Kubernetes operator maturity | Deep-insight operator capability requires SLOs plus alerting rules wired to them |

## Workflows

### Workflow 1: Define a new SLO

1. Pick the user journey to protect (e.g., "checkout completion").
2. Choose SLI type (request-success-rate, latency, availability, freshness, correctness).
3. Define the SLI precisely: numerator/denominator with concrete labels.
4. Pick a target by measuring 30 days of historical SLI value: `target = floor(p50 of last 30 days x 100) / 100`. This avoids targets the system has never sustained.
5. Pick a window (28 days = 4 calendar weeks, recommended).
6. Render the SLO definition with all required fields.
7. Derive burn-rate alert thresholds from the error budget math above.
8. Write the error budget policy (what happens when budget burns).
9. Run through the review checklist — it must pass before the SLO is "live".

### Workflow 2: Quarterly SLO review

1. For every active SLO, run the review checklist — fix any failing findings.
2. Look at last quarter's data: was the SLO too easy (never burned budget)? Tighten it. Too hard (frequently burned)? Loosen the target or fix the system. Did burn-rate alerts fire usefully (not too noisy, not too late)? Adjust thresholds.
3. Audit error budget policies — were they actually followed when budget burned?
4. Record revised SLOs with date stamps; archive old versions.

### Workflow 3: SLO-driven rollback

1. A new deploy starts burning error budget faster than baseline.
2. A burn-rate alert fires.
3. Trigger an auto-rollback via a feature-flag kill switch.
4. Feed the postmortem into the next SLO revision.

## Anti-patterns

- **99.99% on every endpoint** — copy-paste SLOs nobody verified the system can sustain
- **CPU usage as SLI** — system metrics aren't user experience
- **Single-window burn-rate alert** — too noisy if 5-min, too slow if 30-day
- **No error budget policy** — burning budget means nothing without an action
- **SLOs without owners** — no one is responsible; they bit-rot
- **SLOs reviewed once a year** — system characteristics change faster than that
- **SLAs in the SLO doc** — different audience, different stakes; keep them separate
- **SLO target = SLA target** — the SLO must be tighter so you beat your contract before customers notice

## Verifiable success

A team using this discipline well should see:

- Every SLO passing review with zero failing findings
- Every SLO with a documented owner, error budget, burn-rate alerts, and policy
- Burn-rate alerts firing at most ~2 times/month per SLO that's actually hit (signal, not noise)
- Mean time to detect an SLO violation under 30 minutes (multi-window burn-rate alerting working)
- A quarterly SLO review happening every quarter, not annually