51. QoS, DSCP, and traffic shaping

51.1 Why QoS exists

On a congested link, a FIFO queue treats a bulk backup and a voice call identically → the call suffers. QoS = deliberately treating some packets better (lower latency, lower loss, guaranteed rate) at the expense of others. QoS only matters at a congestion point and only helps if the bottleneck is a link you control — it does nothing across the public internet (which ignores your DSCP or remarks it to 0).

51.2 Marking: DSCP and the IP header

The IPv4 DSCP field (6 bits, top of the old ToS byte; IPv6 Traffic Class) + 2 ECN bits. Standard code points (RFC 4594):

Marking happens at the trust boundary (the switch/router port facing an endpoint): either honor what the endpoint set (trusted phones), or classify and re-mark based on ACLs/DPI/app. Internal core routers then just act on the mark.

51.3 Queueing disciplines (what the mark triggers)

• Priority Queuing (PQ) / Low-Latency Queuing (LLQ): EF gets a strict-priority queue (always served first) but policed to a cap so it can't starve everything.
• CBWFQ / DWRR / HTB classes: each class gets a weight → a guaranteed share of the link under congestion, can borrow idle capacity.
• WRED / CoDel / PIE per class: drop/mark early within a class to keep its queue short (and give TCP a congestion signal before the buffer is full).
• Shaping vs policing:
  • Policing = drop (or re-mark) anything over rate now. Bursty, cheap, TCP- unfriendly (causes retransmit storms).
  • Shaping = buffer and release at the target rate (token bucket). Smooth, adds latency, TCP-friendly. Use shaping toward the customer, policing at ingress against abuse.

51.4 Linux tc shaping (the practical bit)

# HTB: cap the interface at 90 Mbit, carve classes:
tc qdisc add dev eth0 root handle 1: htb default 30
tc class add dev eth0 parent 1:  classid 1:1  htb rate 90mbit ceil 90mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 90mbit prio 0  # VoIP
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 50mbit ceil 90mbit prio 1  # interactive
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 20mbit ceil 90mbit prio 2  # bulk/default
# low-latency AQM inside each class:
tc qdisc add dev eth0 parent 1:10 fq_codel
tc qdisc add dev eth0 parent 1:20 fq_codel
tc qdisc add dev eth0 parent 1:30 fq_codel
# classify by DSCP EF -> 1:10
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dsfield 0xb8 0xfc flowid 1:10

Or just use CAKE, which does shaping + fairness + DSCP-aware tins + AQM in one line: tc qdisc replace dev eth0 root cake bandwidth 90Mbit diffserv4 besteffort.

51.5 802.1p (Layer 2 QoS) and where it applies

The 3-bit PCP field in a VLAN tag (Ch. 5) carries a priority class on that L2 segment only — used by switches for per-port egress queues, and by Wi-Fi as WMM access categories (voice/video/best-effort/background) that change the radio backoff timing. DSCP (L3) survives across routers; PCP (L2) is rewritten every hop.

51.6 War story

A branch office's VoIP was fine until 9am, then choppy. The 20 Mbit/s WAN link saturated with cloud-backup and Windows-update traffic each morning; all traffic shared one FIFO on the router, so voice packets queued behind 1500-byte bulk frames — ~50–150 ms of jitter. Fix: on the WAN egress, shape to 19 Mbit/s (just under line rate, to own the queue), put DSCP EF voice in an LLQ capped at 3 Mbit/s, mark updates/backups LE, fq_codel inside each class. Jitter dropped to <10 ms with no bandwidth added. Lesson: QoS is queue management at your bottleneck; shape just below line rate so the queue forms where you can prioritize, and scavenger-mark the bulk traffic.

Try it yourself

tc -s qdisc show dev eth0            # per-class sent/dropped/backlog
tc -s class show dev eth0
# See DSCP on packets:
sudo tcpdump -i eth0 -v 'ip and (ip[1] & 0xfc) != 0' | grep -o 'tos 0x[0-9a-f]*'
# Set DSCP on your own test traffic:
ping -Q 0xb8 <peer>                  # ToS/DSCP EF
iperf3 -c <peer> --tos 184           # 184 = 0xb8 = DSCP 46 (EF)
# Verify it survives the path (often it won't across the internet):
sudo tcpdump -i eth0 -v 'host <peer>' | grep tos