33. News Feed / Timeline

Business context. A social product's home feed. The north-star metric is time spent / DAU retention. The feed is the main surface for ads revenue. Freshness and relevance drive engagement; read latency drives session length.

Requirements. Post (text and media); follow; a home timeline (ranked, paginated); 200M DAU; timeline p99 < 200 ms; a new post appears in followers' feeds within ~5 s; posts are durable. Estimates as in §3.3: ~3.5k writes/s at peak and ~140k timeline reads/s.

Data model.

posts(post_id PK [snowflake: time-sortable], author_id, body, media_refs, created_at)   -- sharded by post_id
follows(follower_id, followee_id)  -- two tables/indexes: by follower and by followee, sharded accordingly
timeline_cache: user_id → list<post_id> (last ~800)   -- Redis/Memcached, sharded by user_id

The central trade-off: fan-out on write vs fan-out on read.

	Fan-out on write (push)	Fan-out on read (pull)
On post	Insert the post_id into every follower's timeline cache	Just store the post
On read	Read one precomputed list → hydrate	Fetch recent posts from everyone followed, merge, rank
Read latency	Excellent	Poor at scale (hundreds of fetches)
Write cost	O(followers) per post; a celebrity with 100M followers = 100M writes	O(1)
Wasted work	Writes to inactive users' timelines	None

Hybrid (the expected answer): push for normal users; pull for celebrities (accounts above ~10k–100k followers). At read time, merge the precomputed timeline with recent posts from the celebrities you follow. Skip fan-out to users inactive for 30+ days, and rebuild their timeline on demand when they return.

Post ─▶ Post service ─▶ posts DB ─▶ outbox/Kafka "post_created"
                                       │
                         Fan-out workers (skip celebrities, skip inactive followers)
                                       ▼
                             timeline cache (per user list)

Read ─▶ Timeline service:
          1. read user's precomputed list (cache)
          2. + recent posts from followed celebrities (pull, cached per celebrity)
          3. candidate set (~500) ─▶ ranking service (ML model, features from feature store)
          4. hydrate top 20 (post cache, user cache, counters) ─▶ response (cursor)

Deep dives.

Ranking: a chronological feed is cheap. A ranked feed needs candidate generation → a light ranker → a heavy ranker, with features (affinity, recency, engagement predictions) and an impression log feeding back into training (§25.1). The ranking stage often has a ~50 ms budget.
Counters (likes, views): never UPDATE posts SET likes = likes + 1 on a hot post. Use sharded counters or in-memory aggregation flushed every N seconds; approximate counts are acceptable.
Media: upload directly to object storage with presigned URLs → async transcoding → CDN. The post goes out only once its media is ready (or with a placeholder).
Deletes and privacy: deleting a post must remove it from timelines quickly. Filter at hydration (check that the post still exists and is visible to this viewer), rather than chasing every cache copy.

Failure modes. Timeline cache loss means rebuilding from pull for active users, which needs a warm-up plan and DB capacity for it. Fan-out backlog during viral events means monitoring lag in seconds and prioritizing the fan-out queue by follower activity.

Strategic lens. Fan-out on write trades storage and write compute (cost) for read latency (engagement). Quantify it: 100M posts/day × an average of 200 active followers = 20B timeline inserts per day (~230k/s). Is that worth it? Yes, because reads outnumber writes 40:1 and latency drives engagement. But skipping inactive users is a direct cost lever worth stating. Ownership: the feed team owns the timeline and ranking; the post service is a platform used by many surfaces.

Follow-ups: how to inject ads (a separate ads ranker, blending rules, pacing); "following" vs "for you" (a recommendation-based candidate source); backfilling a timeline when you follow someone new; GDPR deletion.

🏗️ Flow: post → fan-out → read with hybrid merge.

 POST tweet ─▶ Tweet svc ─▶ Tweet store (Manhattan/MySQL-like, by tweet_id) ─▶ event
                                                                      │
       Fan-out svc: get follower IDs (social graph svc, paginated) ◀──┘
         if author.followers > 100k → STOP (celebrity: pulled at read)
         else for each follower batch (1k ids): skip inactive → LPUSH timeline:{uid} tweet_id; LTRIM 800
 GET home ─▶ Timeline svc:
     ids = LRANGE timeline:{me} 0 800
         + recent tweet ids of celebrities I follow (per-celebrity cache)
     ─▶ candidate pool (+ out-of-network candidates from recommendation svc)
     ─▶ ranker (features: author affinity, engagement prediction, recency)
     ─▶ visibility filter (deleted, blocked, muted, NSFW settings) ─▶ hydrate ─▶ page 1 + cursor

🌍 How it's built in production.

Twitter "Timelines at Scale" (Raffi Krikorian, QCon 2013): home timelines were lists of tweet IDs in Redis, replicated, capped at about 800 entries, built by fan-out on write. High-follower accounts were merged at read time. A single tweet from a celebrity could take minutes to fan out, which is why the hybrid exists.
Facebook News Feed (Multifeed) is described as more pull-oriented: leaf servers hold recent actions per user, and an aggregator queries them at read time and ranks. It's a different point on the same trade-off, driven by heavy ranking and many content types.
Instagram's feed moved from chronological to ranked (2016). That's a product change that turned the feed into a candidate generation plus ML ranking system (§25.1).

⚠️ Corner cases.

Unfollow and block: the precomputed timeline still contains the unfollowed user's posts. Filter at read time (cheap, always correct), and clean up asynchronously.
Account goes private, or a post is deleted: the same answer. Visibility is evaluated at hydration, never trusted from the cached list.
Celebrity threshold oscillation: a user crossing 100k followers flips between push and pull, causing gaps or duplicates. Use hysteresis (push below 90k, pull above 110k) and dedupe at merge.
Timeline cache eviction for a returning user: rebuild on demand by pulling from followees (expensive for users following 5k accounts). Cap it, and show a "catching up" state while warming in the background.
Ranked pagination: the ranking changes between page 1 and page 2, so items repeat or are skipped. Snapshot the ranked candidate list per session (a cursor that references the snapshot).
Duplicate content: the same post via a retweet, a quote, and the original. Dedupe by root post ID in the candidate stage.