32. Reno, NewReno, CUBIC, and BBR
32.1 The family tree
| Algorithm | Signal it reacts to | Increase rule | Decrease on loss | Where it shines / hurts |
|---|---|---|---|---|
| Tahoe (1988) | loss (dup-ACK or RTO) | slow start / linear | → 1 (always slow start) | historical |
| Reno (1990) | loss | AIMD | ×0.5, fast recovery | one loss/window only |
| NewReno (1999) | loss | AIMD | ×0.5, stays in recovery for multiple holes | still 1 repair/RTT without SACK |
| CUBIC (2006, Linux default) | loss | cubic function of time since last loss | ×0.7 (β=0.7) | high BDP, RTT-fairer than Reno; still fills buffers → bufferbloat |
| BBR / BBRv2/v3 (2016–, Google) | measured bottleneck bandwidth & RTT (model-based, mostly ignores loss) | probe BW/RTT in cycles | doesn't halve on loss | high throughput on lossy/long paths, low queueing; v1 could be unfair to CUBIC and starve shallow buffers |
| Vegas (1994) | increasing RTT (delay) | proactive | back off before loss | pure delay-based; loses to loss-based flows |
| DCTCP (2010) | fraction of ECN-marked ACKs | proportional | fine-grained | data-center only (needs ECN + shallow-buffer switches); tiny queues |
| BBR vs CUBIC coexistence | — | — | — | on a shared bottleneck, BBRv1 often takes more than its share; BBRv2/v3 add loss + ECN response to fix this |
| Illinois, YeAH, H-TCP, Scalable, HighSpeed | loss (+ delay hints) | aggressive high-BDP variants | varies | niche / legacy long-fat-network tuning |
32.2 CUBIC in one paragraph
After a loss at window W_max, CUBIC sets a cubic curve
cwnd(t) = C·(t − K)³ + W_max, where K is the time to grow back to W_max.
Just after loss the curve is concave — grows fast toward W_max (recover the
lost ground quickly). Near W_max it flattens (probe cautiously around the
last known ceiling). Past W_max it turns convex — accelerates to look for
new capacity. Because t is wall-clock time, two flows with different RTTs on the
same bottleneck grow similarly → better RTT fairness than Reno. Downside: still
loss-based, so it keeps pushing until the bottleneck buffer overflows —
filling deep buffers and adding latency (bufferbloat, Ch. 44).
32.3 BBR in one paragraph
BBR builds a model of the path: BtlBw (max delivery rate seen) and RTprop
(min RTT seen). It paces sending at BtlBw and keeps ~1 BDP in flight, so it
achieves high throughput without filling the buffer — latency stays near
RTprop. It periodically probes: briefly send faster to test for more
bandwidth (ProbeBW), and every ~10 s send slower to re-measure bare RTT
(ProbeRTT). It largely ignores packet loss as a congestion signal, which is
why it screams on lossy Wi-Fi / trans-oceanic links where CUBIC collapses — but
also why BBRv1 can be aggressive/unfair toward loss-based flows and misbehave in
very shallow buffers. BBRv2/v3 add explicit responses to loss and ECN and are
friendlier.
32.4 Choosing one (Linux)
sysctl net.ipv4.tcp_available_congestion_control # what's compiled/loaded
modprobe tcp_bbr # load BBR
sysctl -w net.ipv4.tcp_congestion_control=bbr # set default
sysctl -w net.core.default_qdisc=fq # BBR wants fq (or fq_codel) for pacing
- Default (CUBIC): fine for most; safe, fair with the internet.
- BBR: great for a CDN/edge/proxy serving users over lossy or long paths; test fairness if you share bottlenecks with your own CUBIC traffic.
- DCTCP: only inside a data center you fully control (ECN end-to-end, AQM switches).
- Per-connection override:
setsockopt(TCP_CONGESTION, "bbr"), or per-routeip route ... congctl bbr.
32.5 Pacing
Bursting a whole cwnd back-to-back causes queue spikes and loss. Pacing
spreads segments over the RTT at roughly cwnd/RTT. BBR requires it; CUBIC
benefits. Linux does it via the fq qdisc or internal TSO-aware pacing
(net.ipv4.tcp_pacing_ss_ratio / tcp_pacing_ca_ratio).
32.6 War story
A backup service in Singapore uploading to us-east-1 (RTT ~230 ms, ~0.3%
loss) maxed at ~25 Mbit/s with CUBIC on a 1 Gbit/s link — the Mathis formula
(Ch. 36) predicts MSS/(RTT·√p) ≈ 1448/(0.23·0.055) ≈ 114 KB/s ≈ ~0.9 Mbit/s
per flow, so even 25 needed dozens of parallel streams. Switching the senders to
BBR (+fq qdisc) took a single stream to ~700 Mbit/s because BBR treats
that 0.3% loss as noise, not a signal to halve. Lesson: on long, slightly
lossy paths, loss-based CC is throughput-limited by √loss; BBR or many parallel
CUBIC streams are the escape hatches.
Try it yourself
ss -tie | grep -oE '(cubic|bbr|reno|dctcp)[^ ]*' # which CC each socket uses
cat /proc/sys/net/ipv4/tcp_congestion_control
# A/B test on one host:
iperf3 -c <server> -t30 -C cubic
iperf3 -c <server> -t30 -C bbr