63. Range requests, streaming, and compression

63.1 Range requests

Server advertises:   Accept-Ranges: bytes
Client asks:         Range: bytes=1000-1999
Server replies:      206 Partial Content
                     Content-Range: bytes 1000-1999/50000
                     Content-Length: 1000

Uses: resumable downloads, video seeking, parallel segment fetching, reading a file header without the whole file.

Detail	Behaviour
`Range: bytes=1000-`	From byte 1000 to the end
`Range: bytes=-500`	The last 500 bytes
Multiple ranges	`multipart/byteranges` response — rarely worth supporting
Unsatisfiable range	`416 Range Not Satisfiable`
`If-Range: <etag or date>`	"Send the range only if the resource is unchanged; otherwise send the whole thing (200)." Essential for correct resumption — without it a resumed download can splice two different versions together.
Weak ETags	Must not be used with `If-Range` (they don't guarantee byte-identity)

63.2 Streaming responses

Three mechanisms, often confused:

Chunked transfer-encoding   HTTP/1.1 body of unknown length; each chunk usable on arrival
HTTP/2 + /3 DATA frames     same effect, native to the protocol
Server-Sent Events          a chunked stream with an event format on top (Ch 62)

Operational gotchas: proxies and CDNs may buffer your stream (nginx proxy_buffering off, X-Accel-Buffering: no), compression can add latency by buffering, and many frameworks buffer the whole response by default.

63.3 Compression

Client:  Accept-Encoding: br, gzip, deflate, zstd
Server:  Content-Encoding: br
         Vary: Accept-Encoding                <-- MANDATORY, or caches serve the wrong body

Codec	Notes
gzip	Universal; good ratio; cheap
brotli (`br`)	~15–20% better than gzip on text; higher CPU at high levels — pre-compress static assets at level 11, use level 4–5 for dynamic
zstd	Fast, increasingly supported
deflate	Historically ambiguous implementations; avoid

Rules:

Never compress already-compressed formats (JPEG, PNG, MP4, zip) — you burn CPU for nothing.
Set a minimum size (~1 KB); compressing tiny responses can make them bigger.
Content-Encoding vs Transfer-Encoding: the former is a property of the resource (and is what ETag covers); the latter is hop-by-hop.
Compression + secrets = BREACH. If a response contains both attacker- controlled input and a secret (like a CSRF token), compression ratio leaks the secret. Mitigate by not reflecting user input alongside secrets, masking tokens per-response, or disabling compression on sensitive endpoints.

63.4 Real-world example

A media API supported Range but not If-Range. A CDN purge mid-download meant a client's resumed request returned bytes from a newer file version, producing corrupted downloads for ~0.3% of users — reported as "random file corruption" for months. Adding If-Range support (falling back to a full 200 on mismatch) fixed it outright.

Try it yourself

curl -sI https://example.com | grep -i accept-ranges
curl -s -r 0-99 https://example.com -o /dev/null -w '%{http_code} %{size_download}\n'
curl -sI -H 'Range: bytes=0-99' https://example.com | grep -i 'content-range\|^HTTP'

# see compression actually applied
curl -sI -H 'Accept-Encoding: gzip, br' https://example.com | grep -i 'content-encoding\|vary'
# compare sizes
curl -s https://example.com | wc -c
curl -s -H 'Accept-Encoding: gzip' --compressed-no-decompress https://example.com 2>/dev/null | wc -c