38. TLS: handshakes, versions, resumption, termination, and offload

TLS (Transport Layer Security) gives an application confidentiality (eavesdroppers see ciphertext), integrity (tampering is detected), and authentication (you're really talking to example.com, and optionally it's really talking to you). It does not change TCP — it is simply the first bytes the application writes into the byte stream, and everything after the handshake is encrypted records. QUIC (Ch. 39, and §38.13 here) reuses the TLS 1.3 handshake but replaces the record layer with its own packet protection.

This chapter, section by section:

• 38.1 Where TLS sits — the record layer
• 38.2 Version history: SSL 2/3 → TLS 1.0/1.1/1.2/1.3 (what changed, what's dead)
• 38.3 Cryptographic building blocks (so the handshakes make sense)
• 38.4 The TLS 1.2 handshake, flight by flight
• 38.5 The TLS 1.3 handshake, flight by flight (+ HelloRetryRequest, key schedule)
• 38.6 0-RTT / early data, and its replay danger
• 38.7 Session resumption: session IDs, session tickets, 1.3 PSK, fleet key management
• 38.8 Certificates, chains, SNI, ECH, OCSP stapling, CT, CAA
• 38.9 The record layer in detail: AEAD, sizing, limits, KeyUpdate, alerts
• 38.10 mTLS (mutual TLS) and client authentication
• 38.11 TLS termination architectures: edge, passthrough, re-encrypt, mesh
• 38.12 TLS offload: software, session cache, kTLS, NIC inline TLS, QAT/HSM
• 38.13 QUIC's use of TLS 1.3
• 38.14 Interactions with TCP that bite
• 38.15 Debugging TLS + war stories

38.1 Where TLS sits — the record layer

[ TCP payload bytes ] = [ TLS record ][ TLS record ][ TLS record ] ...

TLS 1.2 record:  ContentType(1) | LegacyVersion(2) | Length(2) | payload(+MAC or AEAD tag)
TLS 1.3 record:  opaque_type=23 | legacy_version=0x0303 | Length(2) | ciphertext
                 (real content type is the last byte *inside* the decrypted blob)

ContentType: 20 change_cipher_spec, 21 alert, 22 handshake, 23 application_data,
             24 heartbeat (RFC 6520, usually disabled - see Heartbleed)

• One record carries ≤ 2^14 = 16384 bytes of plaintext (max_fragment_length / record_size_limit extensions can lower it).
• Records do not align to TCP segments. A record can span many segments; a segment can hold several records. The receiver must buffer until it has a whole record before it can decrypt/authenticate it — this is why a single lost TCP segment stalls every TLS record queued behind it (TCP head-of-line blocking, §38.14).
• After the handshake, everything (including HTTP/2 frames, close signals, even the real content-type in 1.3) is inside application_data records. On the wire you see only type 23, a length, and ciphertext.

38.2 Version history — what changed and what is dead

What CBC-mode attacks taught the design (why 1.3 is minimal): BEAST, Lucky13, POODLE, and the padding-oracle family all exploited "MAC-then-encrypt" CBC in TLS ≤1.2. Compression attacks (CRIME on TLS compression, BREACH on HTTP compression) killed TLS-level compression. RSA key transport with PKCS#1 v1.5 gave the Bleichenbacher / ROBOT oracles and has no forward secrecy. TLS 1.3 removed every one of these options rather than trusting operators to disable them.

Practical policy today: enable TLS 1.2 + 1.3 only; on 1.2 allow only ECDHE key exchange + AES-GCM/ChaCha20-Poly1305; disable renegotiation, compression, and session-ID caching you don't manage. Test with testssl.sh or SSL Labs.

38.3 Cryptographic building blocks (quick reference)

A cipher suite names the combination. TLS 1.2: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 = ECDHE key exchange, RSA cert auth, AES-128-GCM AEAD, SHA-256 KDF/PRF. TLS 1.3 shortened it to just the AEAD+hash: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256 (key exchange and signature are negotiated in separate extensions).

Forward secrecy (a.k.a. PFS): because ECDHE uses a fresh ephemeral key pair per handshake, an attacker who records ciphertext today and steals the server's long-term private key next year still cannot decrypt it. Static-RSA key transport had no PFS — one stolen key retroactively decrypts years of traffic. This is the single biggest reason to be on 1.3 (or 1.2-ECDHE-only).

38.4 The TLS 1.2 handshake, flight by flight

   Client                                        Server
   ------                                        ------

   ClientHello                    -------->
     client_random, session_id, cipher_suites[], compression=null
     extensions: server_name(SNI), supported_groups, ec_point_formats,
                 signature_algorithms, ALPN, status_request(OCSP),
                 renegotiation_info, [session_ticket], extended_master_secret

                                                 ServerHello
                                                   server_random, chosen cipher, session_id
                                                 Certificate            (server chain)
                                                 [CertificateStatus]    (stapled OCSP)
                                                 ServerKeyExchange      (server ECDHE pubkey,
                                                                         signed with the cert key)
                                                 [CertificateRequest]   (mTLS only)
                                  <--------       ServerHelloDone

   [Certificate]                                  (client chain, mTLS only)
   ClientKeyExchange                              (client ECDHE pubkey)
   [CertificateVerify]                            (client signs transcript, mTLS)
   [ChangeCipherSpec]                             (everything after this is encrypted)
   Finished                       -------->       (PRF over transcript, under new keys)

                                                 [NewSessionTicket]
                                                 [ChangeCipherSpec]
                                  <--------       Finished

   Application Data (HTTP request) <------->      Application Data (HTTP response)

Key computation:

premaster_secret = ECDHE(client_ephemeral_priv, server_ephemeral_pub)   # both sides get the same value
master_secret    = PRF(premaster_secret, "master secret"
                          [ or "extended master secret", session_hash ], client_random + server_random)[0..48]
key_block        = PRF(master_secret, "key expansion", server_random + client_random)
                 -> client_write_MAC_key, server_write_MAC_key (CBC suites only),
                   client_write_key, server_write_key,
                   client_write_IV, server_write_IV

Notes that matter operationally:

• 2 round trips before the client can send the request (ClientHello→ ServerHelloDone is RTT 1; ClientKeyExchange/Finished→server Finished is RTT 2). Plus the TCP handshake = 3 RTT to first byte on a cold connection.
• ServerKeyExchange is signed by the certificate's private key — that's the authentication. If the signature or curve is weak, the handshake is weak even if the AEAD is strong.
• Extended Master Secret (EMS, RFC 7627) binds the master secret to the full handshake transcript, closing the triple-handshake attack. Enable it; some old stacks and middleboxes break resumption without it.
• Renegotiation: either side can restart a handshake mid-connection (client cert step-up, rekey). The 2009 plaintext injection flaw (CVE-2009-3555) was patched with the renegotiation_info extension (RFC 5746); still, disable client-initiated renegotiation (it's a cheap DoS: forces server asymmetric crypto).
• CBC-mode 1.2 suites are still standardized; prefer GCM/ChaCha20 and disable CBC where you can (Lucky13 timing).

38.5 The TLS 1.3 handshake, flight by flight

   Client                                        Server
   ------                                        ------

   ClientHello                    -------->
     client_random, legacy_session_id, cipher_suites=[AES_128_GCM_SHA256, ...]
     extensions:
       supported_versions = [TLS 1.3]            <- the real version lives here
       supported_groups   = [x25519, secp256r1, ...]
       key_share          = x25519 client ephemeral pubkey
       signature_algorithms = [ecdsa_secp256r1_sha256, ...]
       server_name(SNI), ALPN, [pre_shared_key],
       [early_data], [psk_key_exchange_modes], [cookie]

                   (server derives the (EC)DHE shared secret from key_share)

                                                 ServerHello
                                                   server_random, chosen cipher,
                                                   key_share = server ephemeral pubkey
                   (both sides now hold handshake_traffic keys; rest is ENCRYPTED)
                                                 {EncryptedExtensions}
                                                 {CertificateRequest}   (mTLS only)
                                                 {Certificate}          (chain, +OCSP, +SCT)
                                                 {CertificateVerify}    (sig over transcript)
                                  <--------       {Finished}            (HMAC over transcript)
                                                 [ChangeCipherSpec]     (dummy, middlebox compat)

   {Certificate}{CertificateVerify}              (mTLS only)
   {Finished}                      -------->
   -- 1-RTT: client may send Application Data with/after its Finished -->
   <------- 0.5-RTT: server may send Application Data right after its Finished

                                  <--------       {NewSessionTicket} x N   (for resumption)

• 1 round trip to first byte (client can send the HTTP request in the same flight as its Finished). With TCP that's 2 RTT cold, 1 RTT if TCP Fast Open is used, 0 RTT with a PSK + early data (§38.6).
• The certificate and everything after ServerHello is encrypted under handshake keys — a passive observer sees SNI (unless ECH) and little else, not the server cert.
• HelloRetryRequest (HRR): if the client's key_share used a group the server won't use, the server replies HRR naming an acceptable group; the client resends ClientHello with a new key_share. Adds 1 RTT. Tune the client's default group (X25519) to match servers and avoid HRR. Also used to force a cookie for DoS resistance (stateless retry).
• Key schedule (HKDF):

  Early Secret     = HKDF-Extract(0, PSK or 0)
  (binder/ext keys derived here for 0-RTT)
  Handshake Secret = HKDF-Extract(DeriveSecret(Early, "derived"), (EC)DHE)
    -> client_handshake_traffic_secret, server_handshake_traffic_secret
  Master Secret    = HKDF-Extract(DeriveSecret(Handshake, "derived"), 0)
    -> client_application_traffic_secret_0, server_application_traffic_secret_0
    -> exporter_master_secret, resumption_master_secret
  

  Traffic keys/IVs come from HKDF-Expand-Label of the relevant secret. After the handshake, KeyUpdate ratchets the application secret forward (application_traffic_secret_{N+1} = HKDF-Expand-Label(secret_N, "traffic upd")).
• CertificateVerify = the server signs the transcript hash with the cert's private key. That single signature is the entire proof of identity in 1.3 (no ServerKeyExchange).
• Finished = HMAC over the transcript with a key derived from the traffic secret; a mismatch means the handshake was tampered with (downgrade, bit-flip) → decrypt_error alert, connection aborted.

38.6 0-RTT / early data

On a repeat visit where the client holds a PSK (from a prior NewSessionTicket), the client can send early_data (the HTTP request itself) in the first flight, encrypted under a key derived from the PSK — zero round trips to first byte.

ClientHello (+ pre_shared_key, + early_data ext)
{early_data: "GET /home HTTP/1.1 ..."}          <- sent immediately, PSK-encrypted
--------------------------------------------->
                          ServerHello ...(accepts or rejects early_data)... Finished
                          {EncryptedExtensions: early_data}   <- "I accepted it"
<---------------------------------------------
{end_of_early_data}{Finished}  then 1-RTT app data

The replay problem: 0-RTT data has no freshness guarantee. A network attacker can capture the early-data flight and resend it to the server (or to another server in the same cluster sharing ticket keys), N times. TLS itself cannot stop this. Therefore:

• Only send idempotent, side-effect-free requests as early data (GET without auth-state changes). Never POST /transfer-money.
• Servers must implement anti-replay: single-use ticket tracking, or a bounded replay window using the ticket age + a cluster-wide strike register, or restrict 0-RTT to safe methods, or disable it. Cloudflare/CDNs typically allow 0-RTT only for GET/HEAD and strip it for anything with cookies.
• early_data also can't change the negotiated parameters — ALPN and cipher must match the original session.

Benefit is real for latency (a whole RTT on every repeat connection), so it's common on CDNs; just gate it hard on the server side.

38.7 Session resumption — and managing it across a fleet

Full handshakes cost round trips and asymmetric crypto (an ECDSA sign or RSA decrypt per connection). Resumption skips the certificate + key-exchange proof by reusing a secret from a previous session.

Mechanism 1 — Session IDs (TLS ≤1.2, stateful). Server invents a session_id, stores {id → master_secret, cipher, ...} in a server-side cache. Client presents the id in a later ClientHello; server looks it up and both skip to an abbreviated handshake (1 RTT). Problems: the server must hold state (RAM), and in a load-balanced fleet the same server must get the resumed connection (sticky routing) or share the cache (Memcached/Redis) — otherwise it falls back to a full handshake.

Mechanism 2 — Session Tickets (RFC 5077, TLS 1.2; stateless). The server encrypts the session state into an opaque ticket with a server-held Session Ticket Encryption Key (STEK / "ticket key") and sends NewSessionTicket. The client stores the ticket and presents it later; any server that holds the same STEK can decrypt it and resume — no shared cache, no stickiness needed.

Mechanism 3 — TLS 1.3 PSK resumption. Unifies the above: after a handshake the server sends one or more NewSessionTicket messages, each establishing a PSK (identified by the ticket) with an associated ticket_age_add and lifetime. A later ClientHello carries pre_shared_key (+ binder: an HMAC proving the client knows the PSK) and, ideally, a fresh key_share too so the resumed session still has forward secrecy (psk_dhe_ke mode). This is also the gateway to 0-RTT (§38.6).

Fleet key management (the operational core):

• All terminators that might receive a resumed connection must share the current STEK / ticket key set. Distribute out-of-band (config management, a secrets store, nginx ssl_session_ticket_key files, HAProxy tls-ticket-keys, Envoy SDS).
• Rotate frequently — daily or faster. A STEK that never rotates is a forward-secrecy hole: stealing today's STEK lets an attacker decrypt every session that resumed under it, including recorded past traffic (this is exactly the mistake that made Twitter/others' PFS "not really PFS" in ~2013, and what the 2018 discourse around long-lived ticket keys was about).
• Keep a small ring: e.g. accept tickets under keys {N, N-1, N-2}, encrypt new tickets only under N. Roll N on a timer; drop the oldest.
• Cap ticket lifetime (ssl_session_timeout, 1.3 NewSessionTicket lifetime) to hours, not days.
• If you can't manage rotation safely, disabling tickets and taking the full- handshake cost can be the more secure choice.
• CDNs solve this by terminating in one security domain per POP with automated key rotation.

Resumption vs security summary:

38.8 Certificates, chains, SNI, ECH, OCSP, CT, CAA

The chain. The server sends its leaf cert + intermediate(s); the client already trusts a root in its trust store. Validation:

1. Build a path leaf → intermediate(s) → trusted root (signatures verify at each link).
2. Each cert in-date (notBefore/notAfter); clock skew and expiry are the #1 real-world TLS outage.
3. Leaf's Subject Alternative Name (SAN) matches the hostname the client wanted (CN is ignored by modern clients). Wildcards match one label.
4. Key usage / extended key usage allows serverAuth.
5. Not revoked (see OCSP/CRL below).
6. Meets policy: CT (below), pinning if any, min key size, allowed sig algs.

Missing intermediate is a classic: works in browsers (they cache or fetch via AIA) but fails in curl, Java, Go, mobile. Always serve the full chain; test from a clean client.

SNI (Server Name Indication). The client puts the target hostname in a ClientHello extension so one IP + one terminator can serve many certs / many sites (virtual hosting over TLS). In TLS 1.2 and 1.3 alike, SNI is plaintext on the wire — it's the main remaining metadata leak, and what censors and corporate DLP filter on.

ECH (Encrypted Client Hello). Encrypts the sensitive parts of ClientHello (including SNI and ALPN) to a public key the client fetches from the DNS HTTPS/SVCB record (ech= param). The outer ClientHello carries a decoy/"public" name; a fronting server decrypts the inner one. Rolling out (Cloudflare, Firefox); needs DNS + provider support; the whole POP must share the ECH key.

Revocation.

• CRL — big signed list of revoked serials; largely impractical at web scale.
• OCSP — client asks the CA "is serial X still valid?" Adds latency, a privacy leak (CA sees your browsing), and soft-fails (clients proceed if the OCSP responder is down → weak).
• OCSP stapling (status_request) — the server periodically fetches a signed, time-stamped "good" OCSP response from the CA and staples it into the handshake (CertificateStatus in 1.2, an extension in 1.3). No client→CA round trip, no privacy leak. Operational care: if your stapling cache expires and the CA's OCSP is unreachable, and you set OCSP Must-Staple on the cert, every handshake fails hard. Monitor staple freshness; keep a fallback; consider short-lived certs instead of Must-Staple.
• Trend: short-lived certificates (90 days → 47 days coming → hours) make revocation almost moot; automate with ACME.

Certificate Transparency (CT). CAs must log every issued cert to public, append-only logs; the leaf carries SCTs (signed timestamps) proving it was logged, delivered via the cert, a TLS extension, or stapled OCSP. Chrome requires CT. Benefit: you can monitor CT logs (crt.sh, certspotter) for unexpected certs on your domains — early warning of mis-issuance or compromise.

CAA DNS records. example.com. CAA 0 issue "letsencrypt.org" tells CAs which CAs may issue for your domain; a compliant CA refuses otherwise. Cheap defense against mis-issuance.

Pinning. HPKP (HTTP header) is dead (foot-gun: bricked sites). App-level pinning (mobile apps pinning a key or intermediate) is still used but risky — pin the intermediate or a backup key, not just the leaf, and ship an out-of-band update path.

38.9 The record layer in detail

AEAD per record. For each record: nonce = static_IV XOR record_sequence_number (1.3) or explicit/implicit IV (1.2); AEAD_encrypt(key, nonce, plaintext, additional_data = record header). Output = ciphertext ‖ 16-byte tag. Any tampering (or a nonce reuse bug) → tag check fails → fatal bad_record_mac alert. Sequence numbers are implicit (never on the wire) and reset to 0 on each key change; reusing an (key, nonce) pair with GCM is catastrophic (forbidden; a few CVEs came from stacks that did it under concurrency).

Record size vs latency vs CPU.

• Big records (16 KB) → fewer AEAD ops, less framing overhead (~0.4%), best bulk throughput.
• But the receiver can't use any of a record until it's all arrived → under loss/low bandwidth, a 16 KB record adds delay before the app sees the first byte. TTFB-sensitive servers use dynamic record sizing: small records (~1–4 KB) at the start of a connection / after idle, ramping to 16 KB once cwnd is large (nginx ssl_buffer_size, Cloudflare's dynamic sizing, Go's DynamicRecordSizingDisabled=false).
• 1.3 adds optional record padding (hide plaintext lengths) — costs bandwidth and CPU.

Limits and rekeying. Each AEAD has a safe data volume per key (AES-GCM: well below 2^34.5 records / ~24 GB with a 16 KB record before you should rekey; ChaCha20-Poly1305 is roomier). TLS 1.3 KeyUpdate lets either side ratchet keys mid-connection without a full renegotiation; long-lived bulk connections (backups, replication, video) should rekey periodically. key_update_request asks the peer to update too.

Alerts. One-byte level (warning/fatal) + description. Common ones: close_notify (graceful shutdown — its absence is a truncation attack signal; libraries differ on whether they treat a missing close_notify as an error), handshake_failure (no common params), protocol_version, bad_certificate / certificate_expired / unknown_ca, bad_record_mac (tamper or key desync), decrypt_error (bad Finished/sig), unrecognized_name (SNI not served), no_application_protocol (ALPN mismatch), internal_error. In 1.3 alerts after ServerHello are encrypted.

Heartbeat (RFC 6520). A keepalive/PMTU extension. OpenSSL's unbounded memcpy on it was Heartbleed (2014, CVE-2014-0160) — leaked server memory including private keys. Almost everyone disables the heartbeat extension now.

38.10 Mutual TLS (mTLS) / client authentication

Normally only the server proves identity. mTLS makes the client present a certificate too:

• 1.2: server sends CertificateRequest (acceptable CAs, sig algs); client replies with Certificate + CertificateVerify (client signs the handshake transcript with its private key).
• 1.3: same messages, encrypted; can also happen post-handshake (post_handshake_auth) — e.g. only when the client hits a protected route.

Uses: service-to-service auth in a mesh/zero-trust network, API clients, VPNs, device identity, admin access. Operational realities:

• You run a private CA (or SPIFFE/SPIRE, Vault, cert-manager) and must handle issuance, short lifetimes, rotation, and revocation for every workload.
• The terminator validates the client chain against your CA bundle (not the public roots) and typically passes the verified identity upstream as a header (X-Client-Cert, X-SPIFFE-ID) — which the backend must trust only from the terminator (network policy + the terminator stripping any client-supplied copy).
• mTLS + TLS termination interact: if an L7 proxy terminates, the backend can't see the client cert unless the proxy forwards it, or you use passthrough / TLS in TLS, or the mesh sidecar does mTLS and the app speaks plaintext on localhost (Istio/Linkerd model).
• SNI-based routing still works; client-cert-based routing needs the proxy to parse the client cert.

38.11 TLS termination architectures

"Where does the encrypted tunnel end?" determines who can see plaintext, who holds private keys, and what each hop can route on.

Legend:  ==TLS==>  encrypted     -->  plaintext     ==mTLS==>  mutually-authenticated TLS

(A) EDGE / OFFLOAD TERMINATION
  client ==TLS==> [ LB / reverse proxy / CDN ]  --HTTP plaintext-->  backend
     - LB holds the cert + private key, does all crypto
     - LB can route by Host/path/header, add X-Forwarded-For, compress, cache
     - backend is simpler/faster; INTERNAL network must be trusted (or segmented)

(B) TLS PASSTHROUGH  (a.k.a. SNI routing / L4)
  client ============ TLS ============> [ L4 LB ] ============ TLS ============> backend
     - LB never decrypts; peeks at plaintext SNI in ClientHello to pick a backend
     - end-to-end encryption; backend holds the cert; LB can't see HTTP, can't
       add headers, can't do L7 routing or WAF
     - cert/key management is distributed to every backend

(C) RE-ENCRYPT / "TLS bridging"
  client ==TLS(public cert)==> [ LB terminates ] ==TLS(internal cert)==> backend
     - LB decrypts (L7 routing, WAF, observability), then opens a NEW TLS
       connection to the backend using an internal CA
     - two handshakes, two CC loops; most common in regulated environments
     - backend sees the LB as the TLS peer; client identity passed via header or mTLS

(D) SERVICE MESH (sidecar mTLS)
  app --plaintext--> [ sidecar ] ==mTLS==> [ sidecar ] --plaintext--> app
     - app code speaks plaintext to 127.0.0.1; sidecars do mTLS with workload
       identities (SPIFFE). Termination is "everywhere, automatically."

Trade-offs:

PROXY protocol. With L4 passthrough or TCP-mode LBs, the backend would otherwise see the LB's IP as the client. The PROXY protocol (v1 text, v2 binary) prepends one line/header to the TCP stream carrying the real src_ip:port / dst_ip:port before the TLS ClientHello. Backend must be configured to expect it only from trusted LB IPs (otherwise it's a spoofing vector). AWS NLB, HAProxy, Envoy, nginx (proxy_protocol) all support it.

38.12 TLS offload: software, session cache, kTLS, NIC, QAT/HSM

"Offload" = moving the cost of TLS off the application/backend. Layers, cheapest change first:

1. Terminate at a dedicated tier (architectural offload). Architecture (A)/(C) above: the reverse proxy / LB / CDN does handshakes and bulk crypto so app servers don't. This is "TLS offload" in the load-balancer sense. Benefits: fewer places hold private keys, app servers scale on business logic, central cipher policy, central cert automation.

2. Cut handshake cost. The expensive part of a full handshake is the asymmetric op (ECDSA P-256 sign ≈ tens of µs; RSA-2048 decrypt/sign ≈ ~10× that). Reduce the number of full handshakes:

• Session resumption (§38.7) — a resumed handshake is symmetric-only. Aim for a high resume rate; monitor it.
• Keep-alive / connection reuse / HTTP/2 — amortize one handshake over many requests. tcp_slow_start_after_idle=0 and long idle timeouts help (Ch. 34, 37).
• OCSP stapling cache shared across the tier (don't let every node hammer the CA).
• Prefer ECDSA certs over RSA for the leaf (much cheaper signs); you can dual- stack ECDSA + RSA and let the client pick.
• TLS 1.3 (fewer messages, 1-RTT, cheaper).

3. Bulk-crypto offload — kTLS (kernel TLS). After the handshake (done in userspace by OpenSSL/BoringSSL), the app hands the negotiated keys to the kernel via setsockopt(SOL_TLS, TLS_TX/TLS_RX, crypto_info). The kernel then does AEAD on the socket's data path. Wins:

• Zero-copy sendfile() of static files/video straight from page cache → NIC, encrypted inline, without copying through userspace. Big for CDNs (Netflix built kTLS largely for this).
• Fewer syscalls/copies for proxied bulk data.
• Pairs with NIC inline TLS (see next) so the CPU does nothing for bulk records. Support: Linux tls module, nginx ssl_conf_command Options KTLS / build-time, HAProxy, and kTLS-aware stacks. RX-side kTLS is newer/less universal than TX.

4. NIC inline TLS ("TLS offload to hardware"). SmartNICs (Mellanox/NVIDIA ConnectX-6 Dx+, some Intel, Chelsio) implement the record-layer AEAD on the NIC. With kTLS, the kernel programs per-socket keys into the NIC; the NIC encrypts on TX / decrypts+authenticates on RX as packets flow, and handles resync on retransmission/reorder. The host CPU is freed from bulk symmetric crypto entirely — matters at 100 GbE+ video/CDN scale. Falls back to software for the handshake and for edge cases.

5. Asymmetric-crypto accelerators — QAT / HSM.

• Intel QuickAssist (QAT) offloads RSA/ECDSA/DH (and bulk crypto, and compression) to a co-processor. OpenSSL via the qatengine/provider; nginx and HAProxy can use it. Turns handshake-bound TLS front-ends (lots of new connections, low resume rate) from CPU-bound into I/O-bound. Also AES-NI/VAES/AVX-512 and ARM crypto extensions are "offload" in the sense of dedicated CPU instructions — always compile your TLS lib to use them (10–20× AES throughput vs pure software).
• HSM (Hardware Security Module) — PKCS#11 / KMS. The private key never leaves the HSM; the front-end sends the to-be-signed handshake hash and gets a signature back. Used for compliance (FIPS 140-2/3), CA keys, and "keyless SSL" (Cloudflare Keyless: the CDN terminates TLS but the customer keeps the private key on-prem in their HSM; the CDN calls back for each signing op). Adds latency per full handshake (network round trip to the HSM) → resumption matters even more.

6. Session-cache / ticket-key distribution as offload infrastructure. A shared, fast STEK store + high resume rate is effectively an offload: it removes asymmetric ops from most connections. Treat ticket-key rotation as a first-class job (§38.7).

Sizing rule of thumb (order-of-magnitude, one modern x86 core, ECDSA P-256, TLS 1.3): full handshakes ≈ 10k–40k/s/core software, ≈ 100k+/s/core with QAT; bulk AES-GCM with AES-NI ≈ several GB/s/core, ≈ line-rate with NIC inline. So a handshake-heavy edge (many short connections) is asymmetric-bound → resume + QAT/ECDSA; a bulk-heavy edge (video, downloads) is symmetric/CPU-copy-bound → kTLS + NIC offload + sendfile.

What can go wrong with offload:

• kTLS + sendfile + a middlebox that mangles → harder to debug (packets encrypted before your tcpdump on TX; capture on RX or use keylog).
• NIC offload resync bugs under heavy loss/reorder (rare, firmware-dependent).
• QAT/HSM as a single point of failure / latency spike; capacity-plan and have software fallback enabled.
• FIPS builds disable ChaCha20/X25519/Ed25519 in some configs → unexpected cipher/curve negotiation, mobile clients falling back.
• Offloaded checksums already make local captures show "bad checksum" (Ch. 49); offloaded TLS makes local TX captures show ciphertext you can't match to app writes — plan capture points accordingly.

38.13 QUIC's use of TLS 1.3

QUIC (Ch. 39) does not run TLS-over-a-stream. It uses the TLS 1.3 handshake state machine to authenticate and derive keys, but:

• Handshake messages travel inside QUIC CRYPTO frames in special Initial/Handshake packets — there is no TLS record layer.
• ClientHello fits in the first flight; QUIC's own transport parameters (flow control limits, max streams, idle timeout, max_udp_payload_size, original DCID, etc.) are carried in a TLS extension (quic_transport_parameters), so transport + crypto negotiation are one shot.
• Keys derived from the TLS key schedule protect QUIC packets (header protection + AEAD payload protection), not records. There are distinct key phases: Initial (keys derived from a well-known salt + the client's Destination Connection ID — so anyone can decrypt Initial packets; this exists only for version negotiation/PMTU/DoS handling), then Handshake, then 1-RTT (application). The Key Phase bit in the short header signals KeyUpdate ratchets.
• 1-RTT to first byte (same as TLS 1.3 over TCP, but QUIC also carries the request, so no separate TCP handshake) and 0-RTT works the same way as §38.6 (PSK from a prior NewSessionTicket, same replay caveats — QUIC servers additionally enforce the 3× amplification limit until the client's address is validated).
• ALPN is mandatory in QUIC (h3, doq, etc.).
• No ChangeCipherSpec, no downgrade to a record layer, most of the transport header is encrypted → far less middlebox ossification than TLS/TCP.
• Retry / stateless reset: a QUIC server under load can send a Retry packet with a token (like TLS 1.3 HRR + cookie) to validate the client address before committing state.
• Encryption is not optional in QUIC — there is no "plaintext QUIC." The only cleartext is a handful of header bits and the Initial packets (which are "encrypted" with public keys purely for tamper-evidence/versioning).

Practical upshot: your TLS knowledge transfers directly (cipher suites, certificates, SNI/ECH, resumption, 0-RTT risk, mTLS), but "TLS termination" for HTTP/3 means a QUIC-aware terminator that routes on the Connection ID (not the 5-tuple, which changes on migration) and speaks UDP. Load balancers need explicit HTTP/3 support (Ch. 39, 40).

38.14 Interactions with TCP that bite

• Round-trip stack-up. Cold HTTPS = TCP handshake (1 RTT) + TLS handshake (1.3: 1 RTT; 1.2: 2 RTT) before the first request byte → 2–3 RTT to first byte. Resumption (1.2 abbreviated / 1.3 PSK) → 1 RTT. 0-RTT → 0. TCP Fast Open can save the TCP RTT but has poor middlebox support. This is the whole reason CDNs/edge termination exist — make those RTTs short (Ch. 4, 50).
• Head-of-line blocking is TCP's, and TLS inherits it. TLS records must be processed in order; one lost TCP segment blocks decryption of every record behind it, which blocks every HTTP/2 stream on that connection. QUIC fixes this by moving crypto below the streams (Ch. 39).
• Record size vs a lost segment. A 16 KB record split across ~11 segments is undecryptable until the last byte arrives; lose one segment and the whole record waits for the retransmit. Dynamic record sizing (§38.9) mitigates.
• MTU / large ClientHello. Many extensions + hybrid post-quantum key shares (X25519MLKEM768 adds ~1.2 KB) push ClientHello past one segment (>1460 B). Middleboxes/LBs that assume "ClientHello fits in one packet" (for SNI inspection) drop or mishandle it → handshake hangs from some networks only. Ensure multi-segment ClientHello reassembly; watch this as PQC rolls out.
• PMTU black hole (Ch. 8) hits TLS exactly like any bulk TCP flow: handshake (small packets) succeeds, first full-size application_data record stalls forever. Signature is identical; fix is the same (allow ICMP type 3 / MSS clamp).
• close_notify and truncation. A clean TLS shutdown sends close_notify before the TCP FIN. If the peer just sends FIN/RST, a strict client treats the response as truncated (possible truncation attack); a lax one accepts it. Mismatched strictness between client libs and a load balancer that RSTs idle connections → spurious "SSL truncated" / ECONNRESET errors at low rates.
• Renegotiation vs middleboxes (1.2): some proxies choke on mid-stream handshakes; 1.3 removed renegotiation entirely (use KeyUpdate / post-handshake auth).
• TLS version/cipher downgrade. Active attackers (or dumb middleboxes) strip extensions or force old versions. 1.3 bakes a downgrade sentinel into server_random and covers everything with the transcript hash + Finished, so downgrades are detected. Still: disable ≤1.1 so there's nothing to downgrade to.

38.15 Debugging TLS + war stories

Try it yourself

# Full handshake detail, chain, negotiated version/cipher/curve, resumption:
openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null
openssl s_client -connect example.com:443 -servername example.com -tls1_3 -brief </dev/null
openssl s_client -connect example.com:443 -servername example.com < /dev/null 2>&1 \
  | grep -E 'Protocol|Cipher|Server Temp Key|Verify|Session-ID|TLS session ticket|Early data'

# Test a specific version / force downgrade behavior:
openssl s_client -connect host:443 -tls1_2 ; openssl s_client -connect host:443 -tls1_1

# ALPN result (h2 / http/1.1):
openssl s_client -alpn h2,http/1.1 -connect example.com:443 -servername example.com </dev/null 2>&1 | grep ALPN

# OCSP staple present & valid?
openssl s_client -connect example.com:443 -servername example.com -status </dev/null 2>&1 \
  | grep -A17 'OCSP response'

# Resumption actually working? (second call should say "Reused")
( echo; sleep 1 ) | openssl s_client -connect host:443 -servername host -sess_out /tmp/s.pem >/dev/null 2>&1
( echo; sleep 1 ) | openssl s_client -connect host:443 -servername host -sess_in  /tmp/s.pem 2>&1 | grep -E 'Reused|New'

# Timing breakdown (TCP vs TLS vs first byte):
curl -w 'tcp:%{time_connect}  tls:%{time_appconnect}  ttfb:%{time_starttransfer}  total:%{time_total}\n' \
  -o /dev/null -s https://example.com

# Decrypt your own traffic in Wireshark (no private key needed for ECDHE/1.3):
SSLKEYLOGFILE=/tmp/keys.log curl https://example.com     # or set it for Chrome/Firefox
#   Wireshark > Preferences > Protocols > TLS > (Pre)-Master-Secret log filename = /tmp/keys.log

# HTTP/3 / QUIC:
curl -sI --http3 https://cloudflare.com | head -1
dig +short HTTPS example.com            # ech= and alpn=h3 hints

# Full posture audit:
testssl.sh https://example.com          # or the SSL Labs / Mozilla Observatory web tools
nmap --script ssl-enum-ciphers -p 443 example.com

# kTLS in use? (Linux)
grep -r . /proc/net/tls* 2>/dev/null ; ss -ti | grep -i tls ; lsmod | grep '^tls'
# NIC TLS offload capability:
ethtool -k eth0 | grep -i tls           # tls-hw-tx-offload / tls-hw-rx-offload

War story 1 — the un-rotated ticket key. A security review found that a company's nginx fleet had ssl_session_ticket_key pointing at a file baked into the base image 18 months earlier and never rotated. Anyone who had ever pulled that image (contractors, a former employee, a leaked registry) held a key that could decrypt every resumed session across the fleet, retroactively, from packet captures — the "forward secrecy" on the marketing page was fiction for ~60% of connections. Fix: a systemd timer generating a fresh key every 12 h, distributed via the secrets store, nginx reloading with a 3-key ring (keyN, keyN-1, keyN-2), ticket lifetime cut to 8 h. Lesson: session tickets are only as forward-secret as your STEK rotation; treat ticket keys like signing keys.

War story 2 — Must-Staple + a flaky CA. A team enabled OCSP Must-Staple on their cert "for security." Months later the CA's OCSP responder had a 40-minute outage; the servers' staple cache had already expired, so every new TLS handshake failed hard and the site went fully down — a self-inflicted outage worse than the revocation risk it defended against. Fix: removed Must-Staple, kept stapling with a long cache + proactive refresh at 50% of validity + alerting on staple age, and moved to 90-day auto-renewed certs. Lesson: Must-Staple converts "CA availability" into "your availability"; only use it if you can guarantee staple freshness.

War story 3 — PQC ClientHello over a legacy LB. After a client-library bump enabled X25519MLKEM768, handshakes started timing out from exactly one partner's network. Their aging L4 load balancer parsed SNI only from the first TCP segment; the ~1.7 KB ClientHello spanned two segments and the LB dropped connections whose ClientHello wasn't complete in packet one. Short-term: pin that route to X25519 only. Long-term: LB firmware update with multi-segment ClientHello reassembly. Lesson: TLS lives in the TCP byte stream — anything that assumes "the ClientHello is one packet" breaks as handshakes grow (PQC makes this imminent for everyone).

War story 4 — offload made it slower. A video edge added QAT cards expecting a throughput jump; p99 handshake latency got worse under load. QAT was configured for RSA-2048 offload, but the certs were ECDSA P-256 — already cheap in software — so requests queued for a co-processor they didn't need while AES-NI bulk crypto (the actual bottleneck at 40 GbE of video) got no help. Fix: turn off QAT for the asymmetric path, enable kTLS + sendfile + NIC inline TLS for the bulk path; CPU dropped ~35% and p99 normalized. Lesson: profile first — handshake-bound and bulk-bound TLS want different offloads (§38.12).