Part 11 — X-ray of one packet
11.1 The byte budget of one full-size packet
A full-size segment of your response leaving the ALB on a standard 1500-byte-MTU Ethernet link:
Layer Header Bytes Running total on the wire
───── ────────────────────────────────── ───── ─────────────────────────
L1 Preamble + SFD 8
L2 Ethernet header (dst, src, type) 14
L3 IPv4 header 20 ┐
L4 TCP header + timestamps option 32 │ MTU = 1500
L6 TLS record header + tag + type (≈22) (in the payload, amortised over a 16 KB record)
L7 HTTP/2 frame header (9) (in the payload)
Application payload 1448 ┘
L2 FCS 4
L1 Inter-frame gap 12
─────
1538 bytes on the wire for 1448 bytes of TCP payload
Efficiency ≈ 94%. With tiny packets (a 40-byte ACK) the ratio flips: a 64-byte minimum frame plus 20 bytes of L1 overhead carries almost nothing. That's why packets per second (pps), not just bits per second, is a real limit on load balancers, firewalls, and NICs.
Overheads that quietly reduce your MTU:
| Technology | Overhead | Effective inner MTU |
|---|---|---|
| PPPoE (many DSL/fibre ISPs) | 8 B | 1492 |
| VXLAN overlay (K8s, VMware) | 50 B | 1450 |
| IPsec VPN (tunnel mode) | ~50–73 B | ~1400–1430 |
| WireGuard | 60 B (IPv4) / 80 B (IPv6) | 1420 typical |
| GRE | 24 B | 1476 |
11.2 Hop-by-hop: what each device reads and rewrites
Follow the first request packet. ✎ = rewritten; 👁 = read but unchanged; — = not looked at.
| Hop | L2 MACs | L3 Src IP | L3 Dst IP | TTL | L4 ports | L5–L7 (TLS/HTTP) |
|---|---|---|---|---|---|---|
| Laptop sends | laptop → router | 192.168.1.23 | 198.51.100.20 | 64 | 54012 → 443 | encrypted |
| Wi-Fi AP (part of router) | ✎ 802.11 → 802.3 | — | — | — | — | — |
| Home router (route + NAT) | ✎ new frame | ✎ 203.0.113.7 | 👁 | ✎ 63 | ✎ 61001 → 443 | — |
| ISP / Internet routers (×12) | ✎ new frame each hop | 👁 | 👁 | ✎ −1 each | — | — |
| Stateful firewalls along the way | ✎ | 👁 | 👁 | ✎ | 👁 (track state) | — |
| ALB (L7 proxy): connection ENDS | ✎ | — | — | — | ends at 443 | decrypts, reads, routes |
| ALB opens a new connection | new | 10.0.1.15 | 10.0.2.77 | 64 (fresh) | new ephemeral → 8080 | new HTTP request (+ X-Forwarded-For) |
| K8s node / CNI | ✎ (veth) | 👁 | 👁 (or DNAT if via a Service IP) | ✎ | 👁 | — |
| Pod | — | — | — | — | — | your handler reads it |
Three insights in that table that come up in architecture reviews:
- Routers don't care about anything above L3. That's what lets the Internet carry any protocol.
- NAT breaks end-to-end addressing. The server sees
203.0.113.7, shared by your whole household. - An L7 proxy is a full endpoint. Two separate TCP connections, two separate TLS sessions, and the client's identity survives only in headers you deliberately add.