13. Pods: the unit of everything

13.1 Anatomy

A Pod is one or more containers that share a network namespace (same IP, talk via localhost), share volumes, and are scheduled together on one node. Usually one app container plus optional helpers.

apiVersion: v1
kind: Pod
metadata:
  name: hello
  labels: { app: hello }
spec:
  containers:
    - name: web
      image: nginx:1.27-alpine
      ports: [{ containerPort: 80, name: http }]
      resources:
        requests: { cpu: 50m, memory: 32Mi }
        limits:   { memory: 64Mi }

kubectl apply -f hello.yaml
kubectl get pod hello -o wide
kubectl port-forward pod/hello 8080:80
kubectl delete pod hello      # gone forever: nothing recreates a bare Pod

That last line is the point: never run bare Pods in production. Use a controller (Deployment, StatefulSet, Job) that recreates them.

13.2 Pod lifecycle and statuses

Phases: Pending → Running → Succeeded / Failed (plus Unknown when the node is unreachable). The interesting detail is in container states and conditions:

Conditions (kubectl get pod x -o yaml | yq .status.conditions): PodScheduled, Initialized, ContainersReady, Ready. Only Ready pods receive Service traffic.

13.3 Init containers

Run to completion, in order, before app containers start. Use them for one-time setup, not for "wait for the DB" loops: your app should retry connections itself (§11).

spec:
  initContainers:
    - name: fetch-config
      image: busybox:1.36
      command: ["sh", "-c", "wget -qO /work/feature-flags.json http://flags.internal/orbit"]
      volumeMounts: [{ name: work, mountPath: /work }]
  containers:
    - name: app
      image: orbit-orders:dev
      volumeMounts: [{ name: work, mountPath: /etc/orbit }]
  volumes:
    - name: work
      emptyDir: {}

13.4 Sidecars: native sidecar containers (1.29+, GA 1.33)

A sidecar is a helper that runs alongside the app for the pod's lifetime: a log shipper, a proxy, a config reloader. Kubernetes now models it explicitly: an init container with restartPolicy: Always. It starts before the app, keeps running, is restarted if it crashes, and is stopped after the app on shutdown. That fixes two long-standing bugs: Jobs that never complete because the sidecar never exits, and proxies that die before the app finishes draining.

spec:
  initContainers:
    - name: cloud-sql-proxy          # starts first, stays up, stops last
      image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.14.0
      restartPolicy: Always
      args: ["--port=5432", "acme:us-central1:orbit-db"]
      startupProbe:
        tcpSocket: { port: 5432 }
  containers:
    - name: app
      image: orbit-orders:dev
      env: [{ name: DATABASE_URL, value: "postgres://orbit@127.0.0.1:5432/orbit" }]

Common patterns: sidecar (extend the app), ambassador (proxy outbound connections, like the SQL proxy above), adapter (normalize output, e.g. a metrics exporter). From Designing Distributed Systems (Burns) and Kubernetes Patterns (Ibryam & Huß).

13.5 The termination sequence (memorize this)

When a pod is deleted (rollout, scale-down, eviction, node drain):

t=0   Pod marked Terminating. Two things happen IN PARALLEL:
      (a) endpoint removal: EndpointSlice controller removes the pod IP ->
          kube-proxy/ingress/LB update over the next ~1-5+ seconds
      (b) on the kubelet: run preStop hook (if any), THEN send SIGTERM to PID 1
t=?   app must: stop accepting new work, finish in-flight work, close connections, exit 0
t=30s (terminationGracePeriodSeconds, default 30) -> SIGKILL, no matter what

The race between (a) and (b) is why rolling deploys drop requests: the app receives SIGTERM and stops listening before every load balancer has stopped sending it traffic. The fix is to keep serving for a few seconds after SIGTERM:

spec:
  terminationGracePeriodSeconds: 45     # > preStop + drain + shutdown timeout
  containers:
    - name: app
      lifecycle:
        preStop:
          sleep: { seconds: 5 }         # native sleep action (1.30+); no shell needed (works on distroless)

Our Go server also sleeps DRAIN_DELAY after SIGTERM while failing readiness (§4.1). Use one or the other, or both (preStop runs first). The budget must fit: preStop (5s) + DRAIN_DELAY (5s) + SHUTDOWN_TIMEOUT (20s) < terminationGracePeriodSeconds (45s).

War story: a team saw a burst of 502s on every deploy, about 0.3% of requests, for months. The pods shut down cleanly in under a second. That was the problem: they shut down before the cloud load balancer deregistered them, which took up to 10 s. A 15 s preStop sleep made the errors disappear. Cloud LBs (AWS ALB in particular) deregister slowly; size the sleep to your LB.

⚠️ Caveat (verify): feature status by version: native sidecars (beta and on by default in 1.29, GA in 1.33) and the preStop.sleep action (beta and on by default since 1.30). Confirm on your cluster with kubectl explain pod.spec.containers.lifecycle.preStop.sleep and kubectl explain pod.spec.initContainers.restartPolicy. If explain doesn't know the field, the API server doesn't either.