system-design

---
name: system-design
description: Use when designing scalable distributed systems, estimating capacity/QPS/storage, choosing caching/database-scaling/queue strategies, or discussing designs like URL shorteners, feeds, or chat.
category: Design
version: 1.0.0
tools: []
---

# System Design Framework

Structured approach to designing large-scale distributed systems. Apply when architecting new services, reviewing designs, estimating capacity, or working through system design questions (URL shortener, rate limiter, news feed, chat, Twitter/Uber-style systems).

## Core Principle

Start with requirements, not solutions. Jumping to architecture before understanding constraints produces over- or under-engineered systems. Scalable systems are assembled from well-understood building blocks (load balancers, caches, queues, databases, CDNs) — the skill is choosing the right blocks, sizing them with estimates, and owning the tradeoffs each choice introduces.

## Scoring

Goal: 10/10. Score a design by how many of the eight Quick Diagnostic checks it satisfies: score = round(passed / 8 x 10). 9-10 = explicit requirements, real estimates, redundancy, a stated DB-scaling and caching strategy, async via queues, monitoring, and a deployment plan, with tradeoffs named. 5-6 = the design works but skips estimation, redundancy, or operations. <=3 = architecture proposed before requirements or estimates exist. Always state the current score, name the failing diagnostic items, and give the specific fix for each.

## 1. The Four-Step Process

Every design follows four stages: (1) understand the problem and establish scope, (2) propose a high-level design and get buy-in, (3) dive deep into critical components, (4) wrap up with tradeoffs and future improvements. Without this structure, designs either stay too abstract or get lost in premature detail — the four steps invest time proportionally, broad strokes first, depth where it matters.

- Step 1 (~5-10 min): clarifying questions, functional and non-functional requirements, agreed scale (DAU, QPS, storage)
- Step 2 (~15-20 min): high-level diagram with APIs, services, data stores, data flow arrows
- Step 3 (~15-20 min): design the 2-3 hardest or most critical components in detail
- Step 4 (~5 min): tradeoffs, bottlenecks, future improvements
- Never skip Step 1 — ambiguous scope wastes all downstream effort; get explicit agreement on assumptions

Applications: a new-service kickoff doc should cover all four steps before coding (requirements, API contract, data model, capacity estimate, then implementation). An architecture review should walk through the steps sequentially: scope, diagram, deep-dive on the riskiest component, open questions. An incident postmortem can trace a failure through the same lens — which requirement was missed, which block failed, what tradeoff caused it.

## 2. Back-of-the-Envelope Estimation

Use powers of two, latency numbers, and simple arithmetic to estimate QPS, storage, bandwidth, and server count before committing to an architecture. Estimation prevents over-provisioning (wasted money) and under-provisioning (outages under load) — a 2-minute calculation can save weeks of rework.

- Powers of two: 2^10 ~ 1 thousand, 2^20 ~ 1 million, 2^30 ~ 1 billion, 2^40 ~ 1 trillion
- Latency numbers: memory read ~100 ns, SSD read ~100 us, disk seek ~10 ms, same-datacenter round trip ~0.5 ms, cross-continent ~150 ms
- Availability nines: 99.9% = 8.77 hours downtime/year; 99.99% = 52.6 minutes/year
- QPS formula: DAU x actions-per-day / 86,400 seconds; peak is typically 2-5x average
- Storage formula: records-per-day x record-size x retention
- Round aggressively — the goal is order of magnitude, not precision

Worked examples: capacity planning — 100M DAU x 5 actions / 86,400 = ~5,800 QPS average, ~30K peak. Storage budgeting — 500M tweets/day x 300 bytes x 365 days = ~55 TB/year. SLA definition — four nines (99.99%) converts to ~52 minutes of allowed downtime per year.

## 3. Building Blocks

Scalable systems are assembled from a standard toolkit: DNS, CDN, load balancers, reverse proxies, application servers, caches, message queues, and consistent hashing. Each block trades one cost for another (a cache trades freshness for read speed; a queue trades latency for decoupling) — introduce a block only once its specific bottleneck appears; adding all of them up front just multiplies failure modes.

- Load balancers: L4 (transport layer — fast, simple) vs L7 (application layer — content-aware routing)
- Cache layers: client, CDN, web server, application (Redis/Memcached), database query cache
- Cache strategies: cache-aside (app manages), read-through, write-through (synchronous), write-behind (asynchronous)
- Message queues (Kafka, RabbitMQ, SQS): decouple producers from consumers, absorb spikes, enable async processing
- Consistent hashing: distributes keys across nodes with minimal redistribution when nodes change (adding a node moves only ~1/n keys)

Applications: read-heavy workloads use cache-aside Redis in front of the database, caching user profiles with a TTL and invalidating on write. Traffic spikes are absorbed with a message queue between the API and workers (e.g., enqueue image-resize jobs; workers pull at their own pace). Global users are served static assets from a CDN edge while the origin serves only API traffic. Uneven load is handled with consistent hashing for shard assignment.

## 4. Database Design and Scaling

Choose SQL vs NoSQL based on data shape and access patterns; scale vertically first, then horizontally (replication and sharding) once vertical limits are reached. The database is usually the first bottleneck — understanding replication, sharding, and denormalization tradeoffs delays expensive re-architectures and makes growth deliberate.

- Vertical scaling is simpler but has a ceiling; horizontal scaling is harder but nearly unlimited
- Replication: leader-follower (one writer, many readers) for read-heavy workloads; multi-leader for multi-region writes
- Sharding strategies: hash-based (even distribution, hard range queries), range-based (easy ranges, hotspot risk), directory-based (flexible, extra lookup)
- SQL fits ACID transactions, joins, defined schema; NoSQL fits flexible schema, horizontal scale, very high write throughput
- Denormalization trades storage and write complexity for read speed — use when reads dominate and data changes rarely
- Celebrity/hotspot problem: one hot shard needs secondary partitioning or a cache layer

Applications: a read-heavy API uses leader-follower with read replicas (reads to replicas, writes to leader, accept slight lag). User data at scale uses hash-based sharding on user_id (hash(user_id) % num_shards for even, independent shards). An analytics dashboard uses denormalized materialized views, pre-joined and aggregated nightly.

## 5. Common System Designs

Most systems are variations of a small set of well-known designs: URL shortener, rate limiter, notification system, news feed, chat, search autocomplete, web crawler, unique ID generator. A mental library of known designs lets you recognize which pattern a new problem resembles and adapt it rather than inventing from scratch.

- URL shortener: base62 encoding, key-value store, 301 vs 302 redirect tradeoff (caching vs analytics)
- Rate limiter: token bucket or sliding window at the gateway; return 429 with Retry-After
- News feed: fanout-on-write (push at post time) vs fanout-on-read (pull at read time); hybrid for celebrity accounts
- Chat: WebSocket for real-time bidirectional messages, queue for delivery guarantees, heartbeat presence service
- Autocomplete: trie of top-k frequent queries; precompute and cache popular prefixes
- Web crawler: BFS with a URL frontier, politeness (robots.txt, per-domain rate limit), dedup via content hash
- Unique IDs: UUID (simple, no coordination) vs Snowflake (64-bit, time-sortable, datacenter-aware)

Applications: a short-link service base62-encodes an auto-increment ID or hash (e.g. short.ly/a1B2c3 maps to a key-value row). API protection uses a token bucket at the gateway (e.g. 100 tokens/min per key, steady refill, reject with 429). A social feed uses hybrid fanout — precompute feeds for accounts under 10K followers, merge celebrity posts at read time.

## 6. Reliability and Operations

A system is only as good as its ability to stay up, recover, and be observed. Health checks, monitoring, logging, and deployment strategy are first-class design concerns, not afterthoughts — production systems fail in ways diagrams never predict, and operational readiness determines whether a failure is a blip or an outage.

- Health checks: liveness (is the process alive) and readiness (can it serve traffic) — Kubernetes uses both
- Three pillars of observability: metrics (Prometheus, Datadog), logging (ELK, CloudWatch), tracing (Jaeger, Zipkin)
- Deployment strategies: rolling (gradual), blue-green (instant switch between identical environments), canary (small percentage first)
- Disaster recovery: RPO (acceptable data loss) and RTO (acceptable recovery time) drive backup and failover strategy
- Multi-datacenter: active-passive (failover) or active-active (requires data sync and conflict resolution)
- Autoscaling: scale on CPU, memory, queue depth, or custom metrics; always set min and max instance counts

Applications: a zero-downtime deploy uses blue-green with health-check gates, switching to green only after checks pass and keeping blue as an instant rollback. A gradual rollout uses canary with metric comparison — e.g. 5% traffic to the new version, compare errors and latency, then promote or rollback. Data safety is defined by RPO/RTO — RPO of 1 hour implies hourly backups, RTO of 5 minutes implies automated failover.

## Common Mistakes

- Architecture before requirements — solves the wrong problem, misses constraints. Fix: spend the first 5-10 minutes on scope (features, scale, SLA).
- No estimation — provisioning off by orders of magnitude. Fix: estimate QPS, storage, bandwidth before choosing components.
- Single point of failure — one component takes down the system. Fix: redundancy at every layer (multi-server, multi-AZ, multi-region).
- Premature sharding — huge operational complexity before it's needed. Fix: scale vertically first, add read replicas, cache aggressively, shard last.
- Caching without invalidation — stale data causes bugs. Fix: define TTL; cache-aside with explicit invalidation on writes.
- Synchronous calls everywhere — one slow service cascades latency to all callers. Fix: use queues for non-latency-critical paths; set timeouts on sync calls.
- Ignoring hotspots — one shard or key hammered, others idle. Fix: detect hot keys; add secondary partitioning or local caches.
- No monitoring or alerting — users find failures before you do. Fix: instrument metrics, logs, and traces from day one.

## Quick Diagnostic (8 checks — score = passed/8 x 10)

1. Are functional and non-functional requirements listed? If no, the design rests on assumptions — write down features, DAU, QPS, storage, latency and availability SLAs.
2. Is there a QPS and storage estimate? If no, capacity is a guess — use DAU x actions / 86,400 for QPS; records x size x retention for storage.
3. Is every component redundant? If no, single points of failure exist — add replicas, failover, or multi-AZ per component.
4. Is the database scaling strategy defined? If no, you hit a wall under growth — go vertical first, then read replicas, then sharding with a clear shard key.
5. Is there a cache for read-heavy paths? If no, the database takes unnecessary load — add a Redis/Memcached cache-aside layer with a defined TTL.
6. Are async paths using queues? If no, tight coupling causes cascading failures — decouple with Kafka/SQS for jobs, notifications, analytics.
7. Is there a monitoring and alerting plan? If no, you're blind to production failures — define metrics, log aggregation, tracing, alert thresholds.
8. Is the deployment strategy defined? If no, releases are risky — use rolling, blue-green, or canary with a stated rollback plan.