Part 4 — Layer 2: Data Link
Job
Get a frame from one device to the next device on the same link, identify both by MAC address, and detect corrupted frames. L2 never sees past the next hop.
In our story
Before you even clicked, L2 had already done a lot of work, and during the click it does a little more at every hop.
Earlier that evening (bootstrapping the link):
- Association. The laptop's Wi-Fi chip scans, finds the SSID
HomeNet-5G(the router's radio has a MAC/BSSID), and associates. - Authentication (WPA3-SAE + 4-way handshake). The laptop and router prove they both know the passphrase without sending it, and derive per-session encryption keys. Wi-Fi frames are encrypted at L2, so a neighbour sniffing the air sees only ciphertext.
- ARP for the gateway. Once DHCP (Part 5) has given the laptop an IP
address and told it "your gateway is
192.168.1.1", the laptop needs the gateway's MAC to build frames:
Laptop → broadcast ff:ff:ff:ff:ff:ff ARP: "Who has 192.168.1.1? Tell 192.168.1.23"
Router → 3c:22:fb:aa:bb:01 ARP: "192.168.1.1 is at 9c:53:22:00:00:01"
The answer is cached (the ARP table) for minutes, so the click itself doesn't trigger a new ARP request.
At the click: one frame, rebuilt at every hop. The laptop's first outgoing packet to the cloud is wrapped like this:
Wi-Fi hop (802.11 frame, as transmitted over the air):
Addr1 (receiver / BSSID) = 9c:53:22:00:00:01 router radio
Addr2 (transmitter) = 3c:22:fb:aa:bb:01 laptop
Addr3 (final L2 dest) = 9c:53:22:00:00:01 router (gateway)
[ encrypted payload: IP packet src 192.168.1.23 → dst 198.51.100.20 ]
Router WAN → ONT → ISP (Ethernet II frame):
Dst MAC = <ISP BNG's MAC>
Src MAC = <router WAN MAC>
EtherType = 0x0800 (IPv4)
[ IP packet, now src 203.0.113.7 (after NAT) → dst 198.51.100.20 ]
FCS (CRC-32)
... a new frame on every subsequent link ...
Cloud: final hop to ALB
Dst MAC = <ALB ENI MAC> Src MAC = <VPC virtual router MAC>
The IP packet inside stays essentially the same the whole way (only NAT and TTL change it). The L2 envelope is thrown away and rebuilt at every router.
How it works
1. The Ethernet II frame. This is the most common structure in networking:
┌──────────┬─────┬─────────┬─────────┬───────────┬───────────┬──────────────┬─────┬─────┐
│ Preamble │ SFD │ Dst MAC │ Src MAC │ 802.1Q tag│ EtherType │ Payload │ FCS │ IFG │
│ 7 B │ 1 B │ 6 B │ 6 B │ 4 B (opt) │ 2 B │ 46–1500 B │ 4 B │12 B │
└──────────┴─────┴─────────┴─────────┴───────────┴───────────┴──────────────┴─────┴─────┘
L1 sync 0x0800 IPv4 CRC gap
0x86DD IPv6
0x0806 ARP
- MAC address: 48 bits. The first 24 bits are the vendor's OUI. Modern phones and laptops randomise their MAC per network for privacy, which breaks "identify devices by MAC" schemes.
- EtherType tells the receiver which L3 protocol is inside. This is the hand-off point between layers.
- FCS is a CRC-32. A frame that fails the check is silently dropped. L2 doesn't retransmit on Ethernet; TCP (L4) has to notice the gap. Wi-Fi does retransmit at L2, because the air loses so many frames.
- MTU (1500 bytes) is the maximum payload. It's the most important number that crosses layers: IP and TCP size themselves to fit it.
2. Switches learn, flood, and forward.
Frame arrives on port 3 from MAC A, destined for MAC B
1. LEARN: "MAC A lives on port 3" (MAC/CAM table)
2. LOOKUP: do I know where MAC B is?
yes → FORWARD out that one port
no → FLOOD out every port except 3 (B will answer, and I'll learn)
3. Broadcast (ff:ff:ff:ff:ff:ff) → always flood
Everything that receives a given broadcast is in one broadcast domain. ARP, DHCP discovery, and some service discovery depend on broadcast, which is why it doesn't scale: a broadcast domain with 5,000 hosts is constantly busy with ARP traffic.
3. VLANs (802.1Q) add a 4-byte tag with a 12-bit VLAN ID (1–4094), so one physical switch can act as many separate broadcast domains (for example "guest Wi-Fi" and "home"). VLANs are the L2 version of network segmentation.
4. Loops are fatal at L2. Ethernet frames have no TTL. If you create a loop between switches, broadcast frames circulate forever and multiply into a broadcast storm that takes down the whole segment within seconds. Spanning Tree Protocol (STP) blocks redundant links to prevent this. Modern data centres avoid big L2 domains altogether (leaf-spine with L3 routing down to the rack).
5. L2 in the cloud is mostly an illusion. Your VPC looks like Ethernet, but:
- There's no real broadcast. The hypervisor or smart NIC (for example AWS Nitro) answers ARP itself and delivers frames directly.
- The "virtual router" at
.1of every subnet is software in the host. - Kubernetes overlays (Flannel, Calico in VXLAN mode) put a whole L2 frame inside a UDP packet (VXLAN, port 4789) to build a virtual L2 network on top of L3. That costs 50 bytes per packet, which is why pod MTUs are often 1450 instead of 1500.
Try it yourself
# ARP/neighbour table: who's on my LAN and what's their MAC?
arp -a # macOS / Linux
ip neigh show # Linux
# Watch ARP in real time (then run: ping 192.168.1.1 in another terminal)
sudo tcpdump -i en0 -n -e arp # -e prints the MAC (L2) headers
# See the L2 header of every packet to the shop
sudo tcpdump -i en0 -n -e host shop.example.com
# Interface MTU
ifconfig en0 | grep mtu # macOS
ip link show eth0 # Linux
# Linux bridge / switch MAC table (on a Linux router or K8s node)
bridge fdb show | head
What breaks
| Symptom | L2 cause |
|---|---|
| Traffic to an IP goes to the old server for minutes after a failover | Stale ARP caches. The new owner must send a gratuitous ARP (keepalived/VRRP does this) |
| Whole office network dies within seconds of someone plugging in a cable | L2 loop → broadcast storm (STP disabled or misconfigured) |
| Two hosts on "the same subnet" can't reach each other | Different VLANs; trunk port not carrying the VLAN |
| Small requests work, large responses hang | MTU mismatch (for example a VXLAN overlay with MTU 1500 inside). Large frames get dropped |
| Intermittent connectivity, log shows "MAC flapping" | Duplicate MAC (cloned VM) or a loop |
| Device keeps getting a new IP / gets blocked by MAC filtering | MAC randomisation on the client |
War story: "Big JSON responses time out, small ones are fine"
After moving to a new Kubernetes CNI, GET /api/orders?limit=10 worked but
?limit=500 hung until timeout. The overlay used VXLAN (50 bytes of overhead)
but the pod interfaces were still set to MTU 1500. Small responses fit in one
frame. Large responses produced full 1500-byte packets, which became 1550
bytes after encapsulation, exceeded the underlying 1500 MTU, and were dropped,
because the ICMP "too big" messages that should have triggered Path MTU
Discovery were filtered. Fix: set the pod MTU to 1450 (or enable jumbo
frames, 9001 in AWS, on the underlay). Lesson: "works for small payloads,
hangs for large ones" is almost always MTU.
Why it matters to you
- "Same subnet" means "same L2 domain." Hosts can talk directly without a router. Anything that relies on that (multicast discovery, VRRP failover, some legacy clustering) won't work the same way in the cloud, so check for it during migrations.
- Failover designs that move an IP need L2 cooperation (gratuitous ARP). In the cloud you use the provider's API instead (move an Elastic IP or ENI), which takes seconds, not milliseconds. Put that in your RTO.
- MTU is a cross-team contract. Overlays, VPNs, tunnels, and PPPoE all eat into it. Write it down for every network path.
- L2 encryption (WPA3, MACsec) protects only one hop. It doesn't replace TLS.
Go deeper in the series
- Ethernet, ARP, switches, VLANs: TCP/IP guide, Chapters 5–7, with ARP captured live.
- MTU and overlays: TCP/IP Chapter 15 and the tunnels lab in Chapter 54, where a WireGuard tunnel measured exactly 1,420 bytes.
- Bridges and veth pairs, built by hand: TCP/IP Chapter 52.