Chapter 25 — Congestion control: sharing the road · TCP/IP

In one sentence

Nobody tells TCP how fast the network is, so it probes: speed up until packets are lost, back off, and repeat — which is why your download speed graph looks like a sawtooth.

The problem

Flow control (Chapter 24) protects the receiver. But what protects the network? If every sender blasts at full speed, routers' queues overflow, packets are lost, everyone retransmits, and it gets worse — a feedback loop.

This actually happened. In October 1986, the NSFNET backbone's throughput collapsed from 32 kbit/s to 40 bit/s — a 1000x drop — because of exactly this. The fix, invented by Van Jacobson in 1988, is what we still use today.

The idea, in plain language

Driving in fog. You don't know the speed limit, so you accelerate gently until something goes wrong, then slow down sharply, then start accelerating again.

Packet loss is the signal. TCP treats a lost packet as "the network is full".

How it actually works

The sender keeps a second limit alongside the receiver's window, called the congestion window (cwnd). It sends the smaller of the two.

   bytes in flight  <=  min( receiver's window , congestion window )
                            \______________/     \_______________/
                             protects the         protects the
                             RECEIVER              NETWORK

Two phases:

   1. SLOW START  (despite the name, this is the FAST ramp-up)
      Start small (about 10 packets).
      Every round trip, DOUBLE the window.
      10 -> 20 -> 40 -> 80 -> 160 ...
      Keep doubling until a packet is lost, or a threshold is reached.

   2. CONGESTION AVOIDANCE (the careful part)
      Now increase by only ONE packet per round trip.
      Creep upward, probing for more capacity.

   ON PACKET LOSS:
      Loss detected by duplicate ACKs -> cut the window roughly in half
                                          and continue (fast recovery)
      Loss detected by timeout        -> cut all the way back to the start
                                          (something is badly wrong)

The result is the famous sawtooth:

   speed
     ^
     |        /|      /|      /|
     |       / |     / |     / |      <- creep up (congestion avoidance)
     |      /  |    /  |    /  |
     |     /   |   /   |   /   |
     |    /    v  /    v  /    v      <- loss: cut in half
     |   /
     |  /  <- slow start (doubling)
     +-----------------------------------> time

Why this matters to you

1. Short connections never reach full speed. A small web page finishes during slow start. This is why round trips dominate web performance, not bandwidth — and why connection reuse (Chapter 28) matters so much.

2. Distance is brutal. The window grows per round trip, so a connection with 100 ms RTT ramps up 10x slower than one with 10 ms RTT. Same bandwidth, 10x slower to get going.

3. A little loss is catastrophic over distance. For classic loss-based TCP, throughput falls roughly with the square root of the loss rate. Going from 0.01% loss to 1% loss (100x worse) cuts your speed by about 10x. On a long path, even 0.1% loss can cap a single connection at a few Mbit/s.

4. This is why downloads use multiple connections. Each gets its own congestion window, so N connections get roughly N times the share.

The algorithms by name

You'll see these names; you don't need the maths.

Name	Idea	Where
Reno / NewReno	The classic sawtooth described above	Historical, still the baseline
CUBIC	Grows as a cubic curve of time since the last loss — fairer to long-distance connections	Default on Linux, Android, and most of the internet
BBR	Ignores loss; instead measures the bottleneck bandwidth and round-trip time and paces to match	Google, YouTube, many CDNs. Much better on lossy/long paths.

BBR is worth knowing about because it solves point 3 above: on a long, slightly-lossy path where CUBIC gets a few Mbit/s, BBR can get hundreds.

Practice (25 min)

A. See your congestion window live:

# Linux: start a big download, then watch in another terminal
curl -o /dev/null https://speed.cloudflare.com/__down?bytes=100000000 &
watch -n0.5 "ss -tin | grep -A1 cloudflare | head -4"

Watch cwnd: grow. That's slow start happening in front of you.

B. Check which algorithm you're using:

sysctl net.ipv4.tcp_congestion_control          # Linux
sysctl net.ipv4.tcp_available_congestion_control

C. Watch the sawtooth. Add a little loss and download something, then graph throughput in Wireshark (Statistics → I/O Graph):

sudo tc qdisc add dev eth0 root netem loss 0.5%
curl -o /dev/null https://speed.cloudflare.com/__down?bytes=50000000
sudo tc qdisc del dev eth0 root          # UNDO

D. Prove that parallel connections beat one. This is a striking demo:

# one connection
time curl -s -o /dev/null https://speed.cloudflare.com/__down?bytes=50000000

# four connections in parallel
time ( for i in 1 2 3 4; do
    curl -s -o /dev/null https://speed.cloudflare.com/__down?bytes=12500000 &
  done; wait )

Same total bytes. On a long or slightly lossy path, the parallel version is often much faster. That is congestion control, measured.

Common confusions

"Why does TCP need to guess? Can't the network tell it?" There's a mechanism for that (ECN — routers mark packets instead of dropping them), but it needs cooperation across the whole path, so deployment has been slow. Loss remains the universal signal.
"Is packet loss always bad?" For loss-based congestion control, loss is the only signal available. Zero loss forever would mean TCP never learns the limit. A small amount is normal and healthy.
"My connection is 1 Gbit/s but the download is 20 Mbit/s." Very likely congestion control + distance + a little loss, or a window that's too small (Chapter 24). Test with parallel connections to find out which.

Check yourself

What signal does classic TCP use to detect congestion?
Why is it called "slow start" when it doubles every round trip?
Why does a short web page never reach full speed?
Why do download managers use multiple connections?

Chapter 25 — Congestion control: sharing the road