31. Congestion control: slow start and congestion avoidance

31.1 The problem: congestion collapse

October 1986: the NSFNET backbone throughput dropped from 32 kbit/s to 40 bit/s — a 1000× collapse. Cause: senders retransmitting into an already congested network, adding load exactly when they should back off, in a vicious cycle. Van Jacobson's 1988 congestion-control algorithms (slow start, congestion avoidance, fast retransmit) saved the internet and are still the skeleton of every TCP.

Core principle — conservation of packets / ACK clocking: in equilibrium, a sender injects a new packet only when an ACK signals one has left the network. The ACK stream is the clock.

31.2 The variables

• cwnd — congestion window (bytes the sender believes the path can hold).
• ssthresh — slow-start threshold: the cwnd boundary between exponential and linear growth.
• IW — initial window. Historically 1–4 MSS; RFC 6928 raised it to 10 MSS (~14.6 KB), so a small HTTP response often fits in the very first burst.

31.3 Slow start (exponential)

cwnd starts at IW (10 MSS on Linux)
for each ACK received:  cwnd += MSS

Per RTT, cwnd roughly doubles (every segment in the window gets ACKed, each adding an MSS). Growth: 10 → 20 → 40 → 80 … MSS. "Slow" is historical — it starts small, but ramps fast. It ends when:

• cwnd >= ssthresh → switch to congestion avoidance, or
• a loss occurs → set ssthresh = cwnd/2, react (Ch. 29 / below).

31.4 Congestion avoidance (linear / AIMD)

Once cwnd >= ssthresh, grow gently:

per RTT:  cwnd += 1 MSS         (implemented per-ACK as cwnd += MSS*MSS/cwnd)

This is Additive Increase. On a loss: Multiplicative Decrease (cwnd *= 0.5 for Reno). Together, AIMD — the property that makes many TCP flows converge to a roughly fair share of a bottleneck: additive increase nudges everyone up equally; multiplicative decrease cuts the biggest flows the most.

cwnd
 |        /|        /|        /|
 |       / |       / |       / |      <- congestion avoidance (linear rise)
 |      /  |      /  |      /  |
 |     /   |     /   |     /   |
 |    /    v    /    v    /    v      <- loss: cut cwnd in half
 |   /  (slow   
 |  /   start)  
 |_/____________________________________ time
      "TCP sawtooth"

31.5 Reaction to loss, summarized

A timeout is the expensive event: full restart. Everything in modern TCP (SACK, RACK, TLP, PRR) is about avoiding RTOs and recovering via the fast path.

31.6 ECN — congestion signal without loss

With ECN (Explicit Congestion Notification) negotiated, a router experiencing congestion marks the IP header (CE) instead of dropping the packet. The receiver echoes this via the ECE TCP flag; the sender reduces cwnd as if there'd been a loss and sets CWR. Benefit: congestion response without losing/retransmitting data — lower latency, no goodput loss. Needs endpoints + routers to cooperate; historically under-deployed, but L4S (Low Latency, Low Loss, Scalable throughput; RFC 9330) and DCTCP in data centers use fine- grained ECN aggressively. Linux: net.ipv4.tcp_ecn (0 off / 1 request / 2 accept-only).

31.7 Fairness, RTT bias, and why it's imperfect

• RTT unfairness: classic AIMD grows cwnd per RTT, so a short-RTT flow ramps faster and steals bandwidth from a long-RTT flow sharing the same bottleneck. CUBIC reduces (not eliminates) this by making growth a function of time since last loss, not RTT.
• Number-of-flows unfairness: an app opening 8 parallel connections gets ~8× the share of one. (This is literally why browsers open 6 connections per host and why "download accelerators" work — they're not more efficient, just greedier.)
• Loss-based vs delay-based flows don't share fairly (BBR vs CUBIC — Ch. 32).

31.8 War story

A video CDN moved a POP closer to users (RTT 80 ms → 12 ms). Average bitrate rose even though bandwidth was unchanged — because with a 6.7× shorter RTT, each flow's cwnd ramped 6.7× faster through slow start and recovered from loss 6.7× faster, so more of the pipe was actually used. Lesson: TCP throughput is governed by RTT as much as by bandwidth; Rate ≈ cwnd/RTT, and cwnd growth itself is per-RTT. Latency is a throughput feature.

Try it yourself

ss -tie dst <peer>      # cwnd:<segs>  ssthresh:<segs>  bytes_sent  bytes_retrans  send <bps>
sysctl net.ipv4.tcp_congestion_control       # active default (usually cubic)
sysctl net.ipv4.tcp_ecn
nstat -az | grep -Ei 'TCPHystartTrainDetect|TCPHystartDelayDetect|TCPSlowStartRetrans'
# Plot cwnd over time for a transfer:
ss -tie -o state established '( dst <peer> )'   # sample in a loop; or use `tcp_probe`/bpftrace