Chapter 2 — Kernel space vs. user space, and the system call

In one sentence

The kernel is the one piece of software allowed to touch hardware directly, everything else (your shell, your browser, your database) runs in "user space" and must formally ask the kernel to do anything that touches the outside world.

Why this matters

When you read that a vulnerability lets an attacker "escape to the kernel" or "gain root," this is the boundary being crossed. It's also why a crashing application usually doesn't take down your whole machine, but a crashing kernel (a "kernel panic") does.

How it actually works

   THE CPU ITSELF enforces this boundary, via "privilege rings" (or
   "modes") built into the hardware -- it's not just a software policy
   the kernel chooses to follow, it's physically enforced by the
   processor:
     RING 0 / KERNEL MODE   full access: can execute any instruction,
                            touch any memory, talk to any device.
     RING 3 / USER MODE     restricted: cannot execute privileged
                            instructions or touch hardware directly.
   Your shell, your text editor, your web server -- all of it runs in
   user mode. Only the kernel runs in kernel mode.

   SO HOW DOES A PROGRAM READ A FILE, IF IT CAN'T TOUCH THE DISK?
   It makes a SYSTEM CALL (a "syscall"): a formal request to the kernel,
   using a special CPU instruction that deliberately, safely switches the
   processor from user mode to kernel mode, runs a specific bit of kernel
   code to service the request, and switches back.

     your program:  open("data.txt", O_RDONLY)
                         |  (this C library function wraps a syscall)
                         v
     CPU switches to kernel mode (a "trap")
                         |
     kernel: checks permissions, finds the file, sets up a file
             descriptor (Chapter 23), switches back to user mode
                         |
     your program: receives a file descriptor number, continues

   THIS IS SLOWER than a plain function call -- crossing the boundary has
   real, measurable overhead (a "context switch," Chapter 9) -- which is
   exactly why well-written programs try to minimise the NUMBER of
   syscalls they make (buffering writes instead of writing one byte at a
   time, for example).

The syscalls you'll meet constantly, so the names stop being scary:

   read() / write()      move bytes between a program and a file/socket
   open() / close()      get/release a file descriptor
   fork() / exec()       create a new process / replace a process's code
                        (Chapter 6)
   mmap()                 map a file or memory region into your address
                        space directly
   socket() / connect()   networking (a socket is just another kind of
                        file descriptor)
   wait()                 a parent process waiting for a child to finish

Real-world example

   CVE-2016-5195, nicknamed "Dirty COW," is a textbook illustration of
   why this boundary has to be enforced PERFECTLY. It was a RACE
   CONDITION (Chapter 17) in how the Linux kernel handled a specific
   memory-management optimisation called copy-on-write, reachable
   through an ordinary, unprivileged sequence of syscalls
   (`open()`/`mmap()`/`write()`, timed carefully against each other).
   An attacker with only a normal, unprivileged user-space login could
   exploit the race to WRITE TO MEMORY THAT SHOULD HAVE BEEN READ-ONLY --
   including files they had no permission to modify, like the system's
   own password database -- effectively promoting themselves to root.
   The bug had existed, unnoticed, in the kernel for close to a DECADE
   before it was found and fixed in 2016, precisely because the
   kernel/user-space boundary is exactly the kind of code where a subtle
   timing bug is both extremely hard to find and extremely damaging once
   found -- it's the one piece of software every single program on the
   machine implicitly trusts completely.

Try it yourself

# See the actual syscalls a simple command makes -- this demystifies
# "system call" faster than any explanation
strace -c ls > /dev/null      # -c summarises counts per syscall
strace ls 2>&1 | head -20     # or see them one by one, in order

# Watch the boundary in real time: this counts context switches
# (roughly, crossings between your program and the kernel)
/usr/bin/time -v ls > /dev/null 2>&1 || time -v ls  # (flags vary by system)

Build it in Go

Go programs make system calls like any other program, but the Go runtime makes many on your behalf (futex, epoll_pwait, clone, mmap). Run any Go binary under strace -f -c and compare the summary with a C program's. Chapter 73 explains what you're seeing and measures when a goroutine costs an OS thread.

Common mistakes

"User mode is for regular users, kernel mode is for root." No — this is about the CPU's privilege level for a given piece of CODE, not about which human is logged in. Even root's shell runs in user mode; it just has permission to ask the kernel to do more things.
Thinking syscalls are instant. They're fast, but far slower than a normal function call, because of the mode switch. This is why high- performance code (and Chapter 16's buffering) works hard to reduce syscall count.
Confusing "kernel" with "operating system." The kernel is the core; "the OS" usually also includes user-space tools (a shell, system daemons, package managers) built on top of it.

Check yourself

What physically enforces the separation between kernel mode and user mode?
Name four syscalls and what each one does.
Why are syscalls slower than ordinary function calls?
In the Dirty COW example, what made this specific class of bug so dangerous?