Part 4 — Container and Kubernetes security

Containers are a process-isolation trick, not a security boundary by default. Kubernetes is a distributed system whose front door hands out credentials and whose datastore holds every secret. This Part covers what actually isolates a container, how the cluster's trust boundaries work, and how an attacker moves from a pod to the node to the whole cluster — and how you stop each step.

Chapter 15 — Container internals and isolation

In one sentence

A container is a normal Linux process fenced off with namespaces, cgroups, dropped capabilities, and a syscall filter — and every well-known "container escape" is a case where one of those fences was left down.

Where we are

You run workloads in containers. To reason about "what if this container is compromised" (Ch 3) you need to know exactly what separates it from the host and from its neighbours — because the defaults are weaker than most people assume.

How it works

The isolation primitives (all kernel features; a "container" is a bundle of them):

   NAMESPACES  give a process its own view of a global resource:
     pid    -- its own process tree (can't see host PIDs)
     net    -- its own interfaces, routes, iptables
     mnt    -- its own filesystem mounts
     uts    -- its own hostname
     ipc    -- its own SysV IPC / POSIX message queues
     user   -- its own uid/gid MAPPING (container uid 0 -> unprivileged
               host uid) -- the strong one, often NOT enabled
     cgroup, time -- its own cgroup root, clock

   CGROUPS     limit and account resources: CPU, memory, PIDs, IO, devices.
               (Also the mechanism abused in some escapes.)

   CAPABILITIES  split root's power into ~40 pieces. A container process
     runs with a SUBSET. The Docker default set still includes things you
     usually don't want (CAP_NET_RAW, CAP_SETUID, CAP_CHOWN,
     CAP_DAC_OVERRIDE, ...). The dangerous ones:
       CAP_SYS_ADMIN  -- mount, pivot_root, many syscalls -> near-root
       CAP_SYS_MODULE -- load kernel modules -> total host compromise
       CAP_SYS_PTRACE -- inspect/modify other processes (+ hostPID = own host)
       CAP_NET_ADMIN  -- reconfigure networking
       CAP_DAC_READ_SEARCH / CAP_DAC_OVERRIDE -- bypass file perms
       CAP_SYS_RAWIO, CAP_SYS_BOOT, CAP_NET_RAW (ARP spoof, Ch G1-37)

   SECCOMP     a BPF filter on syscalls. Docker's default profile blocks
     ~44 dangerous syscalls (keyctl, ptrace in some modes, mount, unshare
     of certain flags, kexec, bpf, ...). Kubernetes does NOT apply
     RuntimeDefault seccomp unless you ask (per-pod
     `seccompProfile: RuntimeDefault`, or the cluster enables
     SeccompDefault).

   LSM (MANDATORY ACCESS CONTROL)  AppArmor or SELinux profiles constrain
     what files/capabilities/network a process can touch, independent of
     DAC. Docker ships a default AppArmor profile; k8s supports
     `securityContext.appArmorProfile` / SELinux options.

   THE DEFAULT CONTAINER = namespaces (usually NOT user ns) + cgroups +
   ~14 default caps + default seccomp + default AppArmor. That stops a
   casual process. It does NOT stop a determined attacker with the wrong
   securityContext.

The escape classes (memorise these — every one is a securityContext or volume review item):

   1  privileged: true            -> ALL caps, ALL host devices, seccomp &
      AppArmor OFF. Mount the host disk (`mount /dev/sda1 /mnt`), write
      `/mnt/etc/...`, or do the cgroup `release_agent` trick. Instant root
      on the node. NEVER in a normal workload.

   2  hostPath volume             -> mount a host directory into the pod.
      `/` , `/etc`, `/root/.ssh`, `/var/lib/kubelet`, `/var/run/
      containerd.sock` (or docker.sock) -> read node secrets, or create a
      privileged container via the runtime socket -> escape.

   3  hostPID: true               -> see host processes; `/proc/1/root` is
      the host root FS; `nsenter -t 1 -a` (with the caps) = host shell.

   4  hostNetwork: true           -> the node's network stack: reach the
      kubelet API (:10250), the cloud metadata endpoint, node-local admin
      services, bypass NetworkPolicy.

   5  Dangerous capabilities without privileged:
       CAP_SYS_ADMIN + seccomp unconfined  -> cgroup-v1 release_agent
         escape; unshare tricks.
       CAP_SYS_MODULE                       -> `insmod` a rootkit.
       CAP_SYS_PTRACE + hostPID             -> inject shellcode into a host
         process.

   6  Mounted container-runtime socket (docker.sock / containerd.sock /
      crio.sock)  -> you ARE the runtime; create a `--privileged` container
      mounting host `/` -> escape.

   7  Writable procfs/sysfs not masked (`/proc/sysrq-trigger`,
      `/proc/sys/kernel/core_pattern`, `/sys/kernel/...`) -> influence the
      host kernel.

   8  Kernel / runtime CVEs:
       runc CVE-2019-5736 (overwrite the host `runc` binary via
         `/proc/self/exe`), CVE-2024-21626 "Leaky Vessels" (fd/cwd leak),
       CVE-2022-0492 (cgroups release_agent, no caps needed on some
         configs), Dirty Pipe (CVE-2022-0847), Dirty COW.
      -> keep the host kernel and runtime patched; this class is why
         "defence in depth" still matters even with a perfect
         securityContext.

The hardened pod (what "restricted" looks like):

   securityContext (pod + container):
     runAsNonRoot: true
     runAsUser: 65532            # a real non-root uid in the image
     allowPrivilegeEscalation: false     # no setuid gain
     readOnlyRootFilesystem: true        # + an emptyDir for /tmp
     capabilities: { drop: ["ALL"], add: ["NET_BIND_SERVICE"] }  # only if
                                                                 # binding <1024
     seccompProfile: { type: RuntimeDefault }
     # appArmorProfile: { type: RuntimeDefault }   (or a custom profile)
   pod spec:
     hostPID: false   hostIPC: false   hostNetwork: false
     automountServiceAccountToken: false   # unless it calls the API
     no hostPath volumes; volumes are emptyDir / PVC / projected / secret
     hostUsers: false            # user namespace -- container root != host root
   image:
     distroless / minimal, non-root by default, pinned by DIGEST, scanned,
     SIGNED (Ch 13)

Worked example

Escaping via a mounted hostPath, then the review that prevents it.

   The pod spec (a "log shipper" someone wrote):
     volumes:
       - name: varlog
         hostPath: { path: /var, type: Directory }   # <-- way too broad
     containers:
       - name: shipper
         volumeMounts: [{ name: varlog, mountPath: /host-var }]

   ATTACK (shell in the container):
     ls /host-var/lib/kubelet/pods/*/volumes/kubernetes.io~secret/*/
       -> every Secret of every pod on this node, in plaintext.
     cat /host-var/lib/kubelet/pki/kubelet-client-current.pem
       -> the kubelet's client cert -> talk to the API server AS the node
          (system:node:<name>) -> read Secrets, get pods on the node,
          and via the Node authorizer, more.
     echo '<static pod yaml with privileged:true, hostPath / >' \
       > /host-var/.../etc/kubernetes/manifests/  (if /etc is reachable)
       -> kubelet runs it -> full node root, persists across reboot.

   THE FIX (admission + review):
     - Pod Security Admission `restricted` on the namespace -> hostPath is
       forbidden outright; the pod won't schedule.
     - If a genuine host-log use case exists: mount the EXACT file/dir
       read-only (`/var/log/pods`, `readOnly: true`), or better, use a
       DaemonSet with a tightly-scoped mount and a dedicated, minimal SA,
       or ship logs via the CRI/stdout path and a node agent.
     - Kyverno/Gatekeeper policy: deny hostPath except an allowlist of
       exact paths for named DaemonSets in `kube-system`.

Practice (60 min) — local kind cluster

Run a --privileged pod (or with hostPID + nsenter). Get a shell on the node. Then delete it and apply Pod Security Admission restricted to the namespace and confirm the same pod is now rejected at creation.
Use the "bad pods" manifests (BishopFox/badPods) to try each escape class (privileged, hostPath, hostPID, hostNetwork, socket mount, specific caps). For each: does it work? what does restricted PSA say?
Deploy a hardened pod (the spec above). Try to escalate from inside: sudo? (no) write to /? (read-only) see host PIDs? (no) reach metadata? (NetworkPolicy — Ch 19) read the SA token? (automount: false).
Enable user namespaces (hostUsers: false) on a pod; inside, id shows root but cat /proc/self/uid_map shows the host mapping; try a previously-working hostPath write to a host-root-owned file — denied.
Run kubescape scan and kube-bench against the cluster; read the pod-security findings.

Across the series

Build the isolation yourself: the Linux guide's Chapter 44 explains namespaces and cgroups, and Chapter 80 builds a container runtime in Go. It also shows exactly why chroot alone isn't isolation and what pids.max stops.
PID 1 and signal handling inside containers: Linux guide, Chapter 74.

Common mistakes

"It's in a container, so it's isolated." Default containers share the host kernel; a wrong securityContext or a kernel CVE crosses the line.
privileged: true for convenience. For a device, a specific capability, or a mount — grant that, not everything.
Broad hostPath mounts. /, /var, /etc, /proc, and runtime sockets are node-takeover primitives. Mount exact files, read-only, or don't.
Running as root in the container. Combined with a mount or a allowPrivilegeEscalation path, root-in-container becomes root-on-node. runAsNonRoot: true.
No seccomp in Kubernetes. Docker gives you the default profile; Kubernetes does not unless you set seccompProfile: RuntimeDefault.
Ignoring the host. A flawless pod spec doesn't save you from an unpatched runc/kernel. Patch nodes; use minimal node OSes (Bottlerocket, Flatcar, Talos, COS).

Check yourself

Name five Linux namespaces and what each virtualises. Which is the "strong" one and is it on by default?
What does CAP_SYS_ADMIN roughly grant, and why is CAP_SYS_MODULE game over?
List five container-escape classes as securityContext/volume review items.
What does readOnlyRootFilesystem + allowPrivilegeEscalation: false + drop: ["ALL"] + runAsNonRoot buy you together?
Why does a mounted containerd.sock mean node compromise?
Why does patching the host still matter with a perfect pod spec?

(Answers: Appendix G.)