Chapter 16 — Caching, buffering, and disk scheduling

In one sentence

Linux aggressively uses "spare" RAM to cache recently-read and recently- written disk data (the page cache), which makes your system faster and your free -h output more confusing, in roughly equal measure.

Why this matters

More production monitoring false alarms are caused by misreading free -h than almost any other single Linux quirk — and understanding the page cache fixes that instantly and permanently.

How it actually works

   THE PAGE CACHE: whenever the kernel reads data from disk, it keeps a
   copy in RAM (in pages it isn't otherwise using) in case that same
   data is needed again soon -- which, for most real workloads, it often
   is (re-reading the same config file, the same popular database rows,
   the same commonly-loaded library). This is transparent to your
   program: you didn't ask for it, you don't manage it, but it makes
   repeated reads dramatically faster (Chapter 12's hierarchy: RAM vs.
   disk).

   BUFFERING WRITES works similarly, in the other direction (Chapter
   14): writes typically land in the page cache first (as "dirty"
   pages) and get flushed to the actual device by the kernel on its own
   schedule (or when explicitly `fsync()`-ed), batching and reordering
   them for efficiency rather than hitting the disk on every single
   write.

   THE KEY INSIGHT FOR READING `free -h` CORRECTLY: this cached data is
   NOT "wasted" or "unavailable" memory. The kernel will INSTANTLY
   reclaim page-cache pages the moment any process actually needs that
   RAM for something else -- there's no cost or delay to giving it back
   (unless it's DIRTY and needs flushing first). This is why modern
   `free -h` output distinguishes:
     "free"       truly, completely unused memory (often deliberately
                 near zero on a healthy, long-running Linux box -- the
                 kernel would rather use spare RAM for cache than leave
                 it idle)
     "available"  the number that actually matters: free memory PLUS
                 memory that could be reclaimed painlessly from cache if
                 needed -- this is "how much RAM can a new process
                 actually use right now."

   DISK SCHEDULING: the kernel's block layer (Chapter 14) doesn't just
   send I/O requests to the device in the order they arrive -- it can
   reorder and batch them (an I/O SCHEDULER) to reduce costly disk-head
   movement (on spinning disks) or maximise the parallel request queue
   depth (on SSDs/NVMe, which have very different performance
   characteristics and mostly use simpler, low-overhead schedulers like
   `mq-deadline` or `none` rather than the seek-minimising algorithms
   designed for spinning disks).

Real-world example

   "The server shows almost no FREE memory, is it about to run out?"
   is one of the single most common, well-documented FALSE ALARMS in
   Linux operations -- widely discussed across sysadmin communities,
   monitoring-tool documentation (many observability platforms
   explicitly warn users about this in their own docs), and countless
   "why is my Linux server always at 95% memory usage" support threads.
   The near-universal answer: it's the page cache doing exactly what
   it's designed to do -- using otherwise-idle RAM to speed up disk
   access -- and it is NOT memory under pressure. A NAIVE monitoring
   alert configured on raw "free memory" percentage will fire constantly
   on a perfectly healthy, well-utilised Linux server, training whoever
   is on-call to ignore the alert entirely (a real, dangerous "alert
   fatigue" pattern) -- which is exactly why correctly-configured
   monitoring alerts on "AVAILABLE" memory (or, better, on actual signs
   of memory PRESSURE like swap activity or OOM kills, Chapter 13)
   instead of raw "free."

Try it yourself

# See the page cache in action, and the free-vs-available distinction
free -h                       # note "available" is much bigger than "free"

# Watch the page cache grow as you read a large file
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches > /dev/null 2>&1 || true
free -h
dd if=/dev/zero of=/tmp/cachetest bs=1M count=300 2>&1 | tail -1
dd if=/tmp/cachetest of=/dev/null bs=1M 2>&1 | tail -1   # first read
free -h                       # cache usage grew
dd if=/tmp/cachetest of=/dev/null bs=1M 2>&1 | tail -1   # second read --
                                                         # should be MUCH
                                                         # faster (served
                                                         # from cache)
rm /tmp/cachetest

# See your current I/O scheduler
cat /sys/block/*/queue/scheduler 2>/dev/null

Common mistakes

Alerting on raw "free" memory percentage. Almost guaranteed to produce false alarms on a healthy system — use "available" memory, or better, actual pressure signals (swap, OOM events).
Manually dropping caches "to free memory" as a routine practice. It's a legitimate diagnostic tool occasionally, but doing it habitually just makes your NEXT disk read slower, for no real benefit — the cache was helping you.
Assuming a busy-looking iostat means your disk is the bottleneck. Cross-check with CPU and memory numbers too — high disk utilisation with everything else idle is a very different story than high disk utilisation alongside CPU saturation.

Check yourself

What is the page cache, and why does the kernel use "spare" RAM for it rather than leaving it empty?
What's the difference between "free" and "available" memory in free -h output, and which one should you actually monitor?
Why is spinning-disk I/O scheduling different from SSD/NVMe I/O scheduling?