Appendix B — Engineering Blog & Paper Map
Use this table to go from a concept in this guide to how a real company built it. Search the title on the company's engineering blog or in the paper archive. Years are approximate publication dates.
| Concept | Company / source | Title (search this) | Guide § |
|---|---|---|---|
| Business-driven availability trade-off | Amazon | Dynamo: Amazon's Highly Available Key-value Store (SOSP 2007) | 27.7, 10.7, 38 |
| Business-level SLI | Netflix | SPS (stream starts per second); Keeping Netflix Reliable Using Prioritized Load Shedding (2020) | 27.7, 14.5, 16.6 |
| Data-driven sharding change | Slack | Scaling Datastores at Slack with Vitess (2020) | 27.7, 9.7 |
| Estimation anchors | Timelines at Scale (QCon 2013 talk) | 3.6, 33 | |
| Connection-heavy servers | 1 million is so 2011 (2012) | 3.6, 34 | |
| L4 load balancing | Maglev: A Fast and Reliable Software Network Load Balancer (NSDI 2016) | 4.5 | |
| L7 proxy / mesh | Lyft | Envoy announcement and docs (2016) | 4.5 |
| Modular monolith | Shopify | Deconstructing the Monolith (2019) | 4.6, 13.5 |
| Idempotent APIs | Stripe | Designing robust and predictable APIs with idempotency (2017) | 5.6, 39 |
| API versioning | Stripe | APIs as infrastructure: future-proofing Stripe with versioning (2017) | 5.6 |
| Push transport | Uber | Uber's Real-Time Push Platform; Uber's Next Gen Push Platform on gRPC | 5.6, 34, 36 |
| DB engine trade-offs | Uber | Why Uber Engineering Switched from Postgres to MySQL (2016) | 6.6 |
| Storage disaggregation | Amazon | Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases (SIGMOD 2017) | 6.6 |
| Message store evolution | Discord | How Discord Stores Billions of Messages (2017); How Discord Stores Trillions of Messages (2023) | 6.7, 7.7, 34 |
| Postgres sharding | Notion | Herding elephants: lessons learned from sharding Postgres at Notion (2021); The Great Re-shard (2023) | 6.7, 9.7 |
| Postgres sharding | Figma | How Figma's databases team lived to tell the scale (2024) | 6.7, 9.7, 29.6 |
| xid wraparound | Sentry | Transaction ID Wraparound in Postgres (2015) | 6.6 |
| Cache at scale | Meta | Scaling Memcache at Facebook (NSDI 2013) | 7.7 |
| Graph cache | Meta | TAO: Facebook's Distributed Data Store for the Social Graph (ATC 2013) | 10.7 |
| Cache replication | Netflix | EVCache posts | 7.7 |
| Failover gone wrong | GitHub | October 21 post-incident analysis (2018) | 8.5, A.2 |
| Splitting a monolithic DB | GitHub | Partitioning GitHub's relational databases to handle scale (2021) | 8.5 |
| ID-encoded sharding | Sharding & IDs at Instagram (2012) | 9.7, 31 | |
| ID-encoded sharding | Sharding Pinterest: How we scaled our MySQL fleet (2015) | 9.7 | |
| Shard management | Meta | Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications (SOSP 2021) | 9.7 |
| Auto-sharding | Slicer: Auto-Sharding for Datacenter Applications (OSDI 2016) | 9.7 | |
| Tunable consistency | Microsoft | Azure Cosmos DB consistency levels documentation | 10.7 |
| Consistency testing | Jepsen | jepsen.io analyses | 10.7 |
| Log-based messaging | Kafka: a Distributed Messaging System for Log Processing (2011); The Log (Jay Kreps, 2013) | 11.8 | |
| Job queue durability | Slack | Scaling Slack's Job Queue (2017) | 11.8 |
| Priority queue | Meta | FOQS: Scaling a distributed priority queue (2021) | 11.8 |
| Effectively-once | Airbnb | Avoiding Double Payments in a Distributed Payments System (2019) | 11.8, 12.6, 39 |
| Workflow engines | Uber / Netflix | Cadence / Temporal docs; Netflix Conductor (2016) | 12.6, 46 |
| Microservices reversal | Segment | Goodbye Microservices (2018) | 13.5 |
| Serverless → monolith | Amazon Prime Video | Scaling up the Prime Video audio/video monitoring service and reducing costs by 90% (2023) | 13.5, 28.5 |
| Domain-oriented services | Uber | Introducing Domain-Oriented Microservice Architecture (2020) | 13.5 |
| Circuit breakers | Netflix | Hystrix wiki; Performance Under Load (adaptive concurrency limits, 2018) | 14.5, 22.7 |
| Rate limiting & shedding | Stripe | Scaling your API with rate limiters (2017) | 14.5, 32 |
| Overload handling | SRE book, Handling Overload; Addressing Cascading Failures | 14.5 | |
| Retries, timeouts, fallback | Amazon | Builders' Library: Timeouts, retries, and backoff with jitter; Avoiding fallback in distributed systems | 14.5 |
| Chaos engineering | Netflix | Chaos Monkey / The Netflix Simian Army (2011) | 14.5 |
| Hexagonal geo index | Uber | H3: Uber's Hexagonal Hierarchical Spatial Index (2018) | 15.6, 36 |
| Code search | GitHub | The technology behind GitHub's new code search (2023) | 15.6 |
| Time-series compression | Meta | Gorilla: A Fast, Scalable, In-Memory Time Series Database (VLDB 2015) | 15.5 |
| Tracing | Dapper, a Large-Scale Distributed Systems Tracing Infrastructure (2010) | 16.6 | |
| Planet-scale monitoring | Monarch: Google's Planet-Scale In-Memory Time Series Database (VLDB 2020) | 16.6 | |
| Real-time analytics | Meta | Scuba: Diving into Data at Facebook (VLDB 2013) | 16.6 |
| Authorization | Zanzibar: Google's Consistent, Global Authorization System (ATC 2019) | 17.2 | |
| Authorization | Airbnb | Himeji: A Scalable Centralized System for Authorization at Airbnb (2022) | 17.2 |
| Customer-managed keys | Slack | Enterprise Key Management (EKM) | 17.3 |
| Lock service | The Chubby lock service for loosely-coupled distributed systems (OSDI 2006) | 19.8 | |
| Partial partitions | Cloudflare | A Byzantine failure in the real world (2020) | 19.8 |
| Time bugs | Cloudflare | How and why the leap second affected Cloudflare DNS (2017) | 19.8 |
| Global SQL | Spanner: Google's Globally-Distributed Database (OSDI 2012) | 19.4, 10.7 | |
| Active-active | Netflix | Active-Active for Multi-Regional Resiliency (2013) | 20.7 |
| Cells & shuffle sharding | AWS | Builders' Library: Workload isolation using shuffle-sharding; Static stability using Availability Zones | 20.4, 20.7 |
| Cellular migration | Slack | Slack's Migration to a Cellular Architecture (2023) | 20.7 |
| Pods | Shopify | A Pods Architecture To Allow Shopify To Scale / SRECon talks | 27.7, 20.7, 40 |
| Safe deployment | Microsoft | Azure Safe deployment practices | 20.7 |
| Stream processing model | The Dataflow Model (VLDB 2015); MillWheel (VLDB 2013) | 21.6 | |
| Exactly-once joins | Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams (SIGMOD 2013) | 21.6, 44 | |
| Ads warehouse | Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing (VLDB 2014) | 44 | |
| Lake upserts | Uber | Apache Hudi origin posts | 21.6 |
| Streaming platform | Netflix | Keystone posts | 21.6 |
| Metastable failures | Academia + AWS | Metastable Failures in Distributed Systems (HotOS 2021); DynamoDB 2015 summary | 22.3, 22.7 |
| Tail latency | The Tail at Scale (CACM 2013) | 22.2 | |
| Live video herd | Meta | Under the hood: Broadcasting live video to millions (2016) | 22.7, 37 |
| Own storage economics | Dropbox | Scaling to exabytes and beyond (Magic Pocket, 2016) | 28.5, 45 |
| Online migrations | Stripe | Online migrations at scale (2017) | 29.5, 29.6 |
| Safe refactors | GitHub | Scientist: Measure Twice, Cut Over Once (2016) | 29.6 |
| Decision types | Amazon | 2015 Letter to Shareholders (Type 1/Type 2 decisions) | 30.7 |
| ML platform | Uber | Meet Michelangelo: Uber's Machine Learning Platform (2017) | 25.5 |
| Graph embeddings | PinSage: Graph Convolutional Neural Networks for Web-Scale Recommender Systems (KDD 2018) | 25.5 | |
| LLM serving | UC Berkeley | Efficient Memory Management for Large Language Model Serving with PagedAttention (SOSP 2023) | 25.5 |
| Per-title encoding | Netflix | Per-Title Encode Optimization (2015); Dynamic optimizer (2018) | 37 |
| Custom video silicon | Google/YouTube | Argos VCU announcement (2021) | 37 |
| Correct KV stores | Apple | FoundationDB: A Distributed Unbundled Transactional Key Value Store (SIGMOD 2021) | 38 |
| Managed KV reality | Amazon | Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service (ATC 2022) | 38 |
| Rate limiting at the edge | Cloudflare | How we built rate limiting capable of scaling to millions of domains (2017) | 32 |
| Global rate limit service | Lyft | envoyproxy/ratelimit | 32 |
| Notification governance | Air Traffic Controller (ATC) posts | 35 | |
| Multiplayer editing | Figma | How Figma's multiplayer technology works (2019) | 41 |
| Collaborative data structures | Microsoft | Fluid Framework docs | 41 |
| Typeahead | Meta | The Life of a Typeahead Query (2010) | 42 |
| Crawling | Academia | Mercator: A Scalable, Extensible Web Crawler (1999) | 43 |
| Incremental indexing | Caffeine (2010); Percolator (OSDI 2010) | 43 | |
| Sync engine | Dropbox | Rewriting the heart of our sync engine (2020) | 45 |
| Erasure coding | Microsoft | Erasure Coding in Windows Azure Storage (ATC 2012) | 45 |
| Consolidated storage | Meta | Facebook's Tectonic Filesystem (FAST 2021) | 45 |
| Distributed cron | SRE book, Distributed Periodic Scheduling with Cron | 46 | |
| Async platform | Meta | Asynchronous computing at Meta (2023) | 46 |