2. Network Fundamentals That Determine Performance

2.1 TCP handshake and connection reuse cost

A cold TCP connection costs 1 RTT before any bytes of your protocol move. Add TLS 1.2 (2 RTT) or TLS 1.3 (1 RTT, 0-RTT for resumption) and a client hitting a fresh connection per request pays 2-3 RTTs of pure overhead. This is why connection pooling / keep-alive is the single highest-leverage performance change most APIs can make.

Fresh HTTPS/TLS1.2 connection over a 50ms RTT link:
  TCP handshake:   50ms
  TLS handshake:  100ms
  HTTP request:    50ms
  ---------------------
  Total before first byte: 200ms

Reused (keep-alive) connection, same link:
  HTTP request: 50ms   (4x faster, purely from reuse)

2.2 DNS

DNS resolution is often the invisible tail-latency source. A misconfigured resolver, or a client re-resolving on every connection because it ignores TTLs (or a service mesh sidecar doing DNS on every call), silently adds 20-200ms. Under load tests, always resolve DNS once and reuse the IP, unless you're specifically testing DNS behavior (e.g., DNS-based failover, GSLB).

2.3 TCP congestion control & the bandwidth-delay product

Throughput is capped by:

Throughput <= TCP_window_size / RTT

This is why a single TCP connection between two data centers 150ms apart, even on a 10 Gbps link, may only push a few Mbps until the congestion window grows (slow start) or unless you use multiple parallel streams (HTTP/2 multiplexing, multiple connections). This matters directly for load tests: if your load generator opens too few connections over a high-latency link, you will bottleneck the generator, not the server under test.

2.4 Nagle's algorithm & delayed ACK

Nagle's algorithm batches small writes; delayed ACK delays acknowledgments up to ~40ms waiting to piggyback data. Combined, they can produce a notorious ~40ms stall on small, latency-sensitive request/response protocols. Fix: disable Nagle (TCP_NODELAY) on latency-sensitive services — virtually every HTTP server and load tool does this by default, but hand-rolled TCP clients (see the C example in §9.1) must set it explicitly.

2.5 HTTP/1.1 vs HTTP/2 vs HTTP/3

Load testing implication: benchmarking an HTTP/2 API from a client that only opens HTTP/1.1 connections will radically understate real client concurrency behavior (a browser reuses one HTTP/2 connection for 100 requests; your tool must be configured to do the same, or configured to open many connections if you're modeling many distinct clients).

2.6 MTU, fragmentation, and packet loss

Standard Ethernet MTU is 1500 bytes. Large JSON/gRPC payloads that exceed the path MTU get fragmented at the IP layer (or segmented at TCP layer, which is normal and fine) — but on paths with a broken PMTU (common with some VPNs/tunnels), fragmented packets get silently dropped, producing intermittent hangs that look like "random" latency spikes. This is a classic hard-to-diagnose production incident — worth explicitly testing with ping -s 1472 -M do <host> to check for PMTU blackholes before blaming the application.