Frontend Labs
⋯
Read as one page Markdown source ↗

34. Chat & Messaging (WhatsApp / Teams)

Business context. Consumer chat (WhatsApp) cares about reliability and privacy (end-to-end encryption) at minimal cost per user. Enterprise chat (Teams/Slack) cares about retention, search, compliance (eDiscovery, legal hold), and admin control. Ask which one, because the designs diverge significantly.

Requirements. 1:1 and group chat (≤ 1k members; large channels later); online presence; delivery and read receipts; offline delivery and push notifications; multi-device; history sync. 500M DAU, 50 messages/user/day → 25B messages/day (~300k/s average, ~1M/s peak). Delivery p99 < 500 ms when both users are online. No message loss; per-conversation ordering.

Design.

 Device ◀══ WebSocket (persistent) ══▶ Gateway fleet (connection terminators, millions of conns each region)
                                         │  session registry: user_device → gateway_id (Redis / in-memory directory)
                                         ▼
                                Chat service ── 1. assign seq number per conversation
                                         │      2. persist to message store (durable BEFORE ack to sender)
                                         │      3. ack sender ("sent ✓")
                                         ▼
                              Fan-out: for each recipient device
                                 online → route to its gateway → push over socket → device acks ("delivered ✓✓")
                                 offline → store in inbox + push notification (APNs/FCM)

Data model.

messages: PK (conversation_id, bucket), clustering seq DESC → wide-column (Cassandra/ScyllaDB/Bigtable)
          bucket = time window (e.g. 10 days) so a hot or old conversation never becomes an unbounded partition
conversations / members: relational or KV
per-device sync cursor: (device_id, conversation_id) → last_seq_delivered

Deep dives.

  • Ordering: a per-conversation sequence number, assigned by the owner of the conversation's partition (a single writer per conversation, via consistent-hash routing or a sequencer). Clients order by seq, not by timestamp. Clients send a client_msg_id for idempotent retries.
  • Durability before ack: the sender's "✓" means the message is persisted (written to a quorum). Delivery is retried until the device acks; devices dedupe by message id.
  • Multi-device sync: each device keeps a cursor per conversation. On reconnect it asks "give me everything after seq N", so syncing is just reading the log.
  • Group fan-out: small groups fan out on write to each member's devices. Very large channels (100k members) switch to fan-out on read: members pull the channel log, with notifications only for mentions. The same hybrid as §33.
  • Presence: expensive at scale. Heartbeats every 30–60 s; propagate presence only to contacts who currently have the chat open (subscribe on view); accept staleness.
  • Gateway deploys: draining millions of WebSockets. Drain gradually with client reconnect jitter, or a reconnect storm will hit the auth service.
  • E2E encryption (consumer): the server stores ciphertext only (Signal protocol). Search and moderation then must happen on the client, which is a product trade-off to state explicitly.

Strategic lens. For enterprise (Teams): the business needs retention policies, legal hold, eDiscovery, data residency per tenant, and tenant isolation, which pushes toward per-tenant partitioning and cells, a search index per tenant, and an immutable compliance store. For consumer: minimize cost per user (efficient connection handling, minimal server-side storage after delivery). Name the business model first, and then the design follows.

Follow-ups: message edits and deletes (tombstones that propagate through the same log); reactions (small events with a reference); attachments (object storage + presigned URLs); spam in groups; how to roll out a protocol change to a billion clients (version negotiation, server-side compatibility for years).

🏗️ Flow: send a message to a group, with one recipient offline.

 Alice ═ws═▶ Gateway G1 ─▶ Chat svc (owner of conv 77, via consistent hash)
                              1. dedupe client_msg_id
                              2. seq = next_seq(conv 77) = 1042
                              3. write messages(conv77, bucket, 1042) quorum ✓
                              4. ack Alice: {client_msg_id → seq 1042, "sent"}
                              5. members(conv 77) = {Bob, Carol}
                                 Bob → session registry → G3 online → push over ws → Bob acks → "delivered"
                                 Carol → no session → inbox(Carol) += (77,1042) → push notif (APNs/FCM),
                                         collapsed per conversation
 Carol reconnects ═ws═▶ G2: "sync conv77 after seq 1031" ─▶ messages 1032..1042 → render → ack read

🌍 How it's built in production.

  • WhatsApp: Erlang/FreeBSD servers, each holding ~2 million concurrent connections (2012). Messages are stored only until delivered, which keeps storage cost per user minimal. A famously small engineering team served hundreds of millions of users.
  • Discord: messages are partitioned by (channel_id, bucket) where the bucket is a time window, stored in Cassandra and then ScyllaDB. A Rust data service layer with request coalescing in front prevents hot channels (a big server's announcement) from overwhelming partitions. The gateway was built on Elixir/BEAM ("How Discord Scaled Elixir to 5,000,000 Concurrent Users", 2017).
  • Slack real-time messaging (2023 blog): gateway servers hold WebSocket connections, channel servers (assigned via consistent hashing) own channel state and fan-out, and separate presence servers handle presence. It's the same separation shown in the flow above.
  • Uber RAMEN (§5.6): a push channel with acks, used for driver offers that expire in seconds.

⚠️ Corner cases.

  • Membership race: Dave is removed from a group at the same moment a message is sent. Does he get it? Put membership changes in the same ordered log (as system messages with seq numbers), and deliver according to membership as of that seq.
  • Edits and deletes after delivery: they're new events (edit seq 1042) that clients apply. Offline devices receive them on sync. Under E2EE, the "delete for everyone" request is best-effort.
  • Large groups and read receipts: 5,000 members × read receipts is a receipt storm. Aggregate them ("read by 1,203"), batch them, or disable them above a size threshold.
  • Reconnect storm after a gateway deploy or outage: millions of clients reconnect at once, hammering auth and sync. Use exponential backoff with jitter on clients, server-side admission control, and staggered drains.
  • Long-offline devices: a phone offline for 60 days needs 50k messages. Paginate sync newest-first, mark gaps, and let older history load lazily.
  • Clock skew on devices: never order by device timestamps. Display the server seq order, and show the sender's timestamp only as a label.
  • Push notification limits: APNs payload size limits and OS throttling. Send minimal payloads ("new message"), and have the app fetch content.

Generated from real-life-system-design-guide.md · 8 Oct 2026 16:36