20. DNS: resolution, records, caching, and modern transports

DNS turns names into addresses (and much more). It is a distributed, cached, hierarchical database. Almost every "the internet is slow/broken" incident has a DNS chapter.

20.1 The name hierarchy

.                                   <- root
 +-- com.                           <- TLD (top-level domain)
      +-- example.com.              <- second-level (a "zone" you control)
           +-- www.example.com.     <- a record within the zone
           +-- api.example.com.

The trailing dot = the root; www.example.com and www.example.com. are the same. Each zone is authoritatively served by a set of name servers and can delegate sub-zones (e.g. example.com delegates eng.example.com to another team's servers via NS records + glue).

20.2 The players

20.3 A full recursive resolution (cold cache)

Client wants A for shop.example.com:

stub -> recursive:  "A shop.example.com?"  (recursion desired)

recursive -> a ROOT server:      "A shop.example.com?"
root      -> recursive:          "I don't know, but ask .com servers: NS a.gtld-servers.net (+ glue A/AAAA)"

recursive -> a .COM server:      "A shop.example.com?"
.com      -> recursive:          "ask example.com's servers: NS ns1.example.com (+ glue)"

recursive -> ns1.example.com:    "A shop.example.com?"
ns1       -> recursive:          "A shop.example.com = 93.184.216.34, TTL 300"  (authoritative, AA bit set)

recursive caches every referral + the final answer, then:
recursive -> stub:               "93.184.216.34, TTL 300"

Next lookup of anything in .com skips the root; next lookup in example.com skips .com; next lookup of shop.example.com within 300 s is served straight from cache. This is why the first request to a new domain is slow and the rest are instant.

20.4 Record types you must know

20.5 TTL and caching — the operational core

• Every record has a TTL; resolvers must not serve it longer than that.
• Lowering TTL takes effect only after the old TTL expires. To migrate an IP with minimal downtime: drop the TTL to 60 s at least the old-TTL duration in advance, then change the record, then raise the TTL back afterward.
• Negative caching: "no such name" (NXDOMAIN) and "no such record" are cached too, for the SOA minimum field's duration. A typo'd record that returns NXDOMAIN can haunt you for hours.
• Resolvers and OSes and browsers each cache. dig bypasses the OS/browser cache; your app may not.
• Some resolvers clamp absurd TTLs (cap at a day, floor at a few seconds) and prefetch popular records before expiry.

20.6 Transport: UDP first, TCP fallback, then encrypted

• Classic DNS: UDP/53, one query per packet. If the response is too big for the client's advertised EDNS0 buffer (or > 512 bytes without EDNS), the server sets the TC (truncated) bit and the client retries over TCP/53. You must allow TCP/53 — DNSSEC, large TXT, and many CDNs need it.
• EDNS0: an OPT pseudo-record that advertises a larger UDP buffer (commonly 1232 today, to dodge fragmentation — Ch. 15), plus flags and options (ECS = EDNS Client Subnet, which passes a truncated client prefix to authoritative servers for geo-accurate answers).
• DoT (DNS over TLS): port 853. DNS inside TLS. Easy to identify/block by port.
• DoH (DNS over HTTPS): port 443, looks like normal web traffic. Used by browsers (often bypassing your OS resolver and your network's DNS policy — a real operational headache: your split-horizon/internal names stop resolving).
• DoQ (DNS over QUIC): port 853/UDP, no head-of-line blocking.
• DNSSEC ≠ encryption. It's authentication/integrity: authoritative servers sign records (RRSIG), a chain of DS→DNSKEY from the root lets a validating resolver detect tampering. It does not hide queries.

20.7 Split-horizon / split-brain DNS

The same name resolves differently depending on who asks: internal clients get 10.x (private), external clients get the public IP. Implemented with views (BIND), separate resolvers, or "internal" zones. Breaks when: a laptop uses DoH and bypasses the internal resolver; a VPN doesn't push the internal search domain; caching carries an internal answer to the coffee shop.

20.8 Resolution details that bite

• search domains + ndots: resolv.conf search corp.example.com and options ndots:5 mean db becomes db.corp.example.com, and even foo.bar.baz.qux (4 dots < 5) gets search suffixes tried first. In Kubernetes, default ndots:5 makes every external lookup do 4–5 failed queries first — a well-known latency/QPS amplifier. Fix: dnsConfig with ndots:1 or use FQDNs with a trailing dot.
• /etc/hosts and NSS order (/etc/nsswitch.conf: hosts: files dns) win before DNS.
• getaddrinfo vs gethostbyname: the former is dual-stack and applies RFC 6724 sorting; use it.
• Happy Eyeballs (Ch. 18) races the A/AAAA connects, not just the lookups.
• One resolver dead in resolv.conf: default behavior tries them in order with a ~5 s timeout each → every lookup stalls 5 s. options rotate / timeout:1 / attempts:2 help; better, run a local caching resolver.

20.9 War story

A deploy "broke DNS" for a service mesh: p50 request latency jumped 150 ms. tcpdump on port 53 showed five queries per outbound call — svc, svc.ns.svc.cluster.local, svc.svc.cluster.local, svc.cluster.local, svc.example.com — the first four NXDOMAIN. Root cause: code used a bare short name with the default Kubernetes ndots:5. Fix: append the trailing dot to make it a FQDN (svc.ns.svc.cluster.local.), cutting it to one query. Lesson: ndots + search can multiply your DNS load 5× invisibly; always check with a capture.

Try it yourself

dig +trace example.com                       # watch the full root->TLD->auth walk
dig +norecurse @a.root-servers.net example.com   # ask a root server directly
dig example.com A +short
dig example.com HTTPS                          # SVCB/HTTPS record
dig -x 93.184.216.34                           # reverse (PTR)
dig +dnssec example.com                        # see RRSIG; 'ad' flag = validated
dig @1.1.1.1 example.com                        # bypass local resolver
resolvectl status                              # systemd-resolved: per-link servers, DNSSEC, DoT
grep -E 'search|nameserver|options' /etc/resolv.conf
# Measure resolver latency:
for i in $(seq 5); do dig +noall +stats example$i.test @1.1.1.1 | grep 'Query time'; done