Frontend Labs
⋯

30. The Leadership Layer: Decisions, Org, Risk

30.1 One-way and two-way doors

Classify every decision on the board:

  • Two-way doors (cheap to reverse): the choice of cache, internal API shape, instance types. Decide quickly and move on.
  • One-way doors (expensive to reverse): the public API contract, the data model and partition key, the consistency guarantee you promise customers, the primary database, data residency commitments, the event schema other teams consume. Slow down, write it up, get review.

Saying "this is a one-way door, so I'd spend time here; the rest I'd decide quickly" is a strong Staff+ signal.

30.2 Writing the decision down

Design doc / RFC skeleton (Google design-doc culture, Amazon narratives, Microsoft specs all look similar):

1. Context & problem (business outcome, metrics)
2. Goals / non-goals
3. Requirements (functional, NFRs with numbers)
4. Proposed design (diagrams, APIs, data model)
5. Alternatives considered (and why rejected)  ← where senior judgment shows
6. Cross-cutting: security, privacy, cost, observability, operability
7. Rollout & migration plan; rollback
8. Risks & open questions
9. Milestones, staffing

Record individual decisions as ADRs (Architecture Decision Records): context, decision, consequences. Future engineers need the why.

30.3 Trade-off matrix (use it on the whiteboard)

Criteria (weight) Option A: Postgres + replicas Option B: DynamoDB Option C: Spanner
Meets latency SLO (3) ✓ (3) ✓ (3) ✓ (2)
Write scale headroom 5 yrs (3) ✗ needs sharding by year 2 (1) ✓ (3) ✓ (3)
Transactions across entities (2) ✓ (3) ~ limited (2) ✓ (3)
Team expertise (2) ✓ (3) ~ (2) ✗ (1)
Cost at projected scale (2) ✓ (3) ~ (2) ✗ (1)
Lock-in (1) ✓ (3) ✗ (1) ✗ (1)
Weighted total 33 31 26

A narrowly beats B, and the whole margin comes from team expertise, cost, and lock-in. If write growth beats the forecast, the "scale headroom" weight rises and B wins. Say which weight you're least sure of, and what you'd need to learn to settle it. The point isn't the arithmetic. It's showing that the decision is explicit, weighted by business priorities, and revisable.

30.4 Org and execution (especially for EMs and Sr. EMs)

Question the interviewer may ask What a strong answer covers
"How would you staff this?" Team topology: stream-aligned teams per domain, a platform team for shared infrastructure; 2-pizza teams; ownership of each component on the diagram
"How would you sequence it?" A thin vertical slice first (end-to-end MVP), then the riskiest unknowns, then scale. Milestones that each deliver user value.
"What are the risks?" A risk register: technical (unproven tech, scale cliffs), dependency (other teams, vendors), people (key-person risk, hiring), with owners and mitigations
"How do you know it's working?" Product metrics + SLOs + DORA metrics (deploy frequency, lead time, change failure rate, time to restore)
"Team disagrees with your design?" Disagree-and-commit with data; time-boxed spikes or prototypes; escalate the decision, not the conflict; written trade-offs
"Legacy system nobody wants to own?" Explicit ownership, a strangler plan, investment justified by incident cost and velocity drag
"On-call is burning people out?" Measure pages per shift; SLO-based alerting only; fix the top-N noisy alerts; error-budget policy; staffing ratio

30.5 Communicating with executives

Translate architecture into outcomes, cost, risk, and time:

"We can hit the Black Friday target with 30% headroom if we ship the read cache and the inventory reservation change by October 15. That costs about $40k per month in extra infrastructure. The main risk is the payment provider's rate limit, and we've asked them to raise it. If they can't, we fall back to queued checkout, which costs an estimated 2% conversion."

30.6 🏗️ Flow: how a significant design decision moves through a company

 Problem identified (incident trend, scale forecast, business initiative)
    │
    ▼
 1-pager / problem statement ─▶ align with stakeholders on goals & non-goals (is this worth solving?)
    │
    ▼
 Design doc / RFC (§30.2) ─▶ async review (comments) ─▶ design review meeting (decide, not present)
    │                                       │
    │          ┌────────── open questions → time-boxed spike / prototype / load test
    ▼          ▼
 Decision recorded (ADR): chosen option, rejected options, one-way doors, owners, revisit triggers
    │
    ▼
 Execution: milestones with exit criteria ─▶ launch review (SLOs, runbooks, rollback) ─▶ retro

30.7 🌍 In production

  • Amazon Type 1 / Type 2 decisions: Jeff Bezos's 2015 shareholder letter separates one-way-door decisions ("Type 1", which should be made slowly and carefully) from two-way doors ("Type 2", which should be made quickly by small groups). Many organizations reuse the framing.
  • Amazon six-page narratives and PR/FAQ ("working backwards"): designs and products start as prose, read in silence at the start of the meeting. Expect these ideas in Amazon loops.
  • Google design docs: a lightweight template (context, goals and non-goals, design, alternatives considered, cross-cutting concerns) reviewed asynchronously. Written decision-making is a large part of Staff+ work at Google.
  • Uber and others' RFC processes: engineering-wide RFCs for changes that affect many teams, with designated approvers per domain.
  • Spotify "squads and tribes": often cited for team topology. Spotify itself has said the model was aspirational and changed over time. Use it as a reference for why org design matters, not as a template.

30.8 ⚠️ Corner cases in technical leadership

  • The rewrite trap: "let's rewrite it from scratch" stops feature work for years while the old system keeps changing underneath. Prefer strangler migrations (§29.1) with value delivered at every step.
  • Second-system effect: the replacement tries to fix everything at once and never ships. Scope v2 to the top 3 problems.
  • A decision without an owner: everyone reviewed it and nobody is accountable. Name a single DRI per decision and per component.
  • Platform by mandate: forcing teams onto an immature internal platform breeds resentment and workarounds. Win adoption by making the platform the easiest path (paved road), and mandate only for safety or compliance.
  • Sunk-cost migrations: a migration is 70% done after 2 years and the last 30% is the hardest. Re-evaluate explicitly. Sometimes the right call is to finish with extra staffing; sometimes it's to stop and live with both systems knowingly.
  • Hero culture: one engineer understands the critical system. That's a key-person risk, so the fixes are documentation, pairing, and on-call rotation (an EM responsibility).

Generated from real-life-system-design-guide.md · 8 Oct 2026 16:36