49. Walkthrough: Design a CDN

The prompt. "Design a content delivery network." Variants: "Design CloudFront / Azure Front Door / Akamai", "Design a global caching layer for our static and video content", "How does a CDN purge content worldwide in seconds?" It's an infrastructure problem about routing, caching hierarchy, global config/purge distribution, TLS at scale, and multi-tenancy. The CDN guide in this wiki has a runnable topology for most of these mechanisms. This walkthrough is about presenting the design in an interview.

Peak egress 100 Tbps = 12.5 TB/s; avg object (blend of web + large files) ≈ 100 KB
  → ~125M requests/s peak globally; largest PoPs ~3-5M req/s
Edge server: ~40-100 Gbps sustained with TLS on modern NICs/CPUs
  → a 5 Tbps PoP needs ~80-120 servers (+ N+1 headroom)
Cache storage: hot working set per large PoP: hundreds of TB (NVMe) + RAM for hottest objects
Logs: 125M req/s × ~500 B/log line ≈ 60 GB/s ≈ 5 PB/day raw
  → cannot ship everything: aggregate at the edge (per-customer counters/minute), sample raw logs,
    ship full logs only for customers who pay for them (compressed, batched)
Billing: per-customer byte counters at each server, aggregated per minute → exact & cheap
Certificates: ~5M customer hostnames → millions of certs at every PoP (lazy-load by SNI)
Config: ~1M customer properties × ~10 KB ≈ 10 GB → must be replicated to every PoP

Control plane (customers):
POST /v1/properties {hostnames[], origins[{host, weight}], rules[{match, ttl, cache_key, headers}]}
PUT  /v1/properties/{id}/versions/{v}/activate        (staging → production, versioned)
POST /v1/purge {urls[] | tags[] | prefixes[] | all:true} → {purge_id}   GET /v1/purge/{id} → progress
GET  /v1/analytics?property=…&metric=hit_ratio&interval=1m

Data plane: plain HTTP(S); origin returns Cache-Control, Surrogate-Key: "product-42 category-7"

                               ┌──────────── GLOBAL CONTROL PLANE ─────────────┐
 Customer API/console ────────▶│ Config svc (validate, version, staged rollout) │
                               │ Purge svc (sequence, fan-out, track acks)      │
                               │ Cert svc (ACME issuance + renewal, §56)        │
                               │ Analytics/billing pipeline                     │
                               └─────────┬───────────────────────────┬──────────┘
                     distribution tree   │ (config, purge, certs)    │ usage, sampled logs
          ┌──────────────────────────────┼───────────────┐           │
          ▼                              ▼               ▼           │
 ┌─────────────── PoP (one of ~300) ──────────────────────────────────────────────────┐
 │ Routers (BGP anycast) ─▶ L4 LB (ECMP + consistent hash, XDP DDoS drop)             │
 │   ─▶ Edge servers: TLS (SNI → cert), HTTP/1/2/3, WAF, rate limit, config lookup    │
 │        cache key → "owner" server in this PoP (consistent hashing) → RAM/NVMe      │
 │        miss ─▶ request collapsing ─▶ upper tier                                    │
 │   local config KV · purge log applier · counters                                   │
 └───────────────┬────────────────────────────────────────────────────────────────────┘
                 │ miss
                 ▼
        Regional tier / origin shield (few, large caches; one per origin region)
                 │ miss (collapsed; warm, pooled, long-lived connections)
                 ▼
            Customer origin

Deep dive 1: routing users to the right PoP

🎤 Candidate: I'd use anycast for resilience and DDoS absorption, and manage load by withdrawing or prepending announcements from overloaded PoPs and by using different anycast prefixes per region, with DNS steering between prefixes. Real-user measurements continuously check whether the routing actually delivers low latency.

Deep dive 2: caching inside and across PoPs

• Within a PoP: if every server caches everything, a 100-server PoP stores each object up to 100 times and effective capacity shrinks 100×. So hash the cache key to an owner server (consistent hashing, §9), and let the server that received the request fetch from the owner over the PoP's fast network. Very hot objects are replicated to every server (detected by request rate) to avoid hot spots.
• Admission policy: a large share of objects are requested only once ("one-hit wonders"). Caching them evicts useful objects. Akamai described cache-on-second-hit using Bloom filters. Modern alternatives are TinyLFU-style admission or S3-FIFO eviction.
• Tiered caching: edge → regional/shield → origin. Hit ratios compound: a 90% edge and 80% shield hit ratio gives 98% offload (1 − 0.1 × 0.2). See CDN guide §5 for measured numbers.
• Request collapsing at both tiers: 10,000 concurrent misses for the same object become one origin fetch (the §7.3 stampede fix applied at the edge).
• Serve stale: stale-while-revalidate (fast responses during refresh) and stale-if-error (origin down → serve the cached copy). This is often the biggest availability feature customers get.
• Large files and video: cache by range slices (e.g. 1 MB chunks), so a seek doesn't need the whole file, and the first byte doesn't wait for a multi-GB origin fetch.

Deep dive 3: global purge in about a second

 Customer POST /purge {tags:["product-42"]} ─▶ Purge svc: assign seq=8,812,331, persist
   ─▶ fan-out tree: control plane → ~10 regional relays → ~300 PoPs → all servers (persistent streams)
   ─▶ each server: apply purge → ack(seq)
   ─▶ purge status = "complete" when all PoPs ack (offline PoPs tracked separately)

 HOW a tag purge is applied cheaply — generation numbers (no scanning):
   tag_gen["product-42"] = 17 → 18       (O(1) update)
   each cache entry stores the generations of its tags at fill time {product-42: 17}
   on lookup: entry.gen(product-42)=17 < current 18 → treat as MISS (lazy invalidation)

• Lazy invalidation by generation turns "purge 2 million objects tagged X" into a single counter increment per server.
• Ordering: purges carry sequence numbers, and edges apply them in order.
• The offline-PoP problem: a PoP disconnected during a purge must not serve content on reconnect until it has caught up to the latest purge sequence (or it drops the affected portion of its cache). Otherwise it serves legally removed content.
• Purge vs in-flight fill race: a fill that started before the purge completes afterwards and stores stale content. Fix: record the generation or purge sequence at fill start, and discard the result if a purge happened meanwhile.

Deep dive 4: TLS for millions of customer domains

• Issuance: automate it with ACME (domain validation via HTTP-01 or DNS-01 through CNAME delegation) and renew at about 2/3 of lifetime. The certificate service is the subject of §56.
• Selection: SNI → certificate lookup in a local index. Load lazily, because millions of certs won't fit in every server's memory; keep an LRU cache of parsed certificates.
• Private key protection: keys encrypted at rest and decrypted only in memory on edge servers. For customers who won't share keys (banks), keyless TLS: the edge performs the handshake but sends the private-key signing operation to the customer's key server.
• Handshake cost: TLS session resumption, with ticket keys rotated frequently and shared within a PoP, plus ECDSA certificates (cheaper to sign than RSA).

Deep dive 5: multi-tenancy and DDoS

• Volumetric attacks: anycast spreads the attack across all PoPs; drop packets early in XDP/eBPF on the L4 layer; SYN cookies.
• Application-layer attacks: per-customer rate limits and bot scoring at the edge.
• Cache fairness: one customer's huge catalog can evict everyone else's content. Use per-customer or per-tier cache quotas (partitioned LRU), with priority for paying tiers. Isolation here is a pricing feature.
• CPU fairness: expensive WAF rules for one customer are capped by per-customer CPU budgets.

Interviewer follow-ups & strong answers

🌍 Real-world references

• Akamai, "The Akamai Network: A Platform for High-Performance Internet Applications" (2010): the foundational architecture paper (mapping system, tiered distribution, overlay routing). Maggs & Sitaraman, "Algorithmic Nuggets in Content Delivery" (2015): cache-on-second-hit via Bloom filters, consistent hashing, and load balancing at Akamai.
• Cloudflare Quicksilver: replicates configuration to every edge server worldwide within seconds. Unimog (2020): Cloudflare's L4 load balancer.
• Fastly: surrogate keys and instant purge are core product features. The June 8, 2021 outage: a valid configuration change by one customer triggered a latent bug and caused errors across most of Fastly's network for under an hour. That's the "global config" failure domain (§14.3), and why config is staged.
• Netflix Open Connect: the internal-CDN variant, with appliances embedded in ISPs and proactive overnight fills.
• This wiki's CDN guide: runnable Go and nginx implementations of multi-tier caching, consistent hashing, purge by surrogate key, signed URLs, mTLS to origin, and multi-CDN failover.

⚠️ Corner cases

• Vary explosion: Vary: User-Agent creates thousands of variants per URL and drops the hit ratio. Normalize to a few device classes.
• Query-string ordering and tracking parameters: ?a=1&b=2 vs ?b=2&a=1, and utm_*. Normalize them in the cache key.
• Set-Cookie on cacheable responses: caching it leaks one user's session to others. Strip it, or don't cache.
• Range-request first-byte latency on huge files: use slice-based caching.
• Clock skew between edge and origin affecting Expires: prefer Cache-Control: max-age, which is relative.
• Negative caching: caching an origin 404 for too long after content is published. Use short negative TTLs and purge on publish.
• Long-lived connections during route changes: an anycast shift moves a client's packets mid-connection to a PoP that doesn't know the connection. Mitigations: stable routing, connection-tracking forwarding between PoPs, or QUIC connection IDs for HTTP/3.