23. Resources, QoS, and right-sizing

23.1 Requests vs limits

resources:
  requests: { cpu: 250m, memory: 256Mi }    # 250 millicores = 1/4 core
  limits:   { memory: 256Mi }               # memory limit = request; NO cpu limit (see below)

23.2 QoS classes (who gets evicted first under node memory pressure)

23.3 The CPU-limit debate

A CPU limit is enforced by the CFS quota in 100 ms periods. A Go service with 8 busy goroutines and limits.cpu: 1 can burn its entire 100 ms quota in 12.5 ms of wall time, then freeze for 87.5 ms. p99 latency spikes appear even though average CPU usage looks low.

# fraction of CFS periods in which the container was throttled
sum by (pod) (rate(container_cpu_cfs_throttled_periods_total{namespace="orbit"}[5m]))
  / sum by (pod) (rate(container_cpu_cfs_periods_total{namespace="orbit"}[5m]))

Widely adopted guidance for latency-sensitive services:

• Always set CPU requests (scheduling + fair share under contention).
• Omit CPU limits for latency-sensitive services on clusters you control, or set them generously (2–4× request). Unused CPU on the node is then usable, and contention is resolved by request-proportional shares.
• Always set memory limits, equal to the memory request. Memory isn't compressible: overcommit leads to node-level OOM, which kills random pods.
• Multi-tenant clusters with untrusted neighbors may require CPU limits. Enforce that with LimitRange.

23.4 Make the runtime container-aware

env:
  # Go 1.25+ sets GOMAXPROCS from the cgroup CPU limit automatically; without a limit, it uses
  # node cores. On older Go, use go.uber.org/automaxprocs.
  - name: GOMEMLIMIT          # soft memory target: GC works harder before the hard limit -> fewer OOMKills
    value: "230MiB"          # ~90% of a 256Mi limit
  # JVM:    -XX:MaxRAMPercentage=75   (container-aware since JDK 10)
  # Node:   --max-old-space-size=200  (in MB; Node does not size the heap from the cgroup on its own)
  # Python: watch out for per-worker memory x worker count (gunicorn --workers)

War story: a Node.js service with a 512 Mi limit was OOMKilled every few hours. V8 sized its heap from the node's 64 GB RAM, so it saw no reason to collect garbage until the cgroup killed it. Setting --max-old-space-size=400 fixed it. The JVM had the same problem before JDK 10.

⚠️ Caveat (verify): runtime behavior differs by version. Go 1.25+ derives GOMAXPROCS from the cgroup CPU limit (older Go needs automaxprocs). Newer Node.js releases take container memory into account when sizing the heap, so the war story applies mainly to older Node versions. Check yours: node -e "console.log(v8.getHeapStatistics().heap_size_limit/1048576)" inside the container.

23.5 Guardrails per namespace

apiVersion: v1
kind: LimitRange
metadata: { name: defaults, namespace: orbit }
spec:
  limits:
    - type: Container
      defaultRequest: { cpu: 100m, memory: 128Mi }   # applied when a container omits requests
      default:        { memory: 256Mi }               # applied when it omits limits
      max:            { memory: 4Gi }
---
apiVersion: v1
kind: ResourceQuota
metadata: { name: orbit-quota, namespace: orbit }
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.memory: 40Gi
    pods: "200"
    services.loadbalancers: "0"    # force traffic through the shared Gateway
    persistentvolumeclaims: "20"

23.6 Right-sizing with data

Requests should come from observed usage, not guesses:

# p95 CPU usage per container over 7 days -> a candidate CPU request
quantile_over_time(0.95, (sum by (container) (rate(container_cpu_usage_seconds_total{namespace="orbit",container!=""}[5m])))[7d:5m])
# max working-set memory over 7 days -> memory request/limit plus ~20-30% headroom
max_over_time(max by (container) (container_memory_working_set_bytes{namespace="orbit",container!=""})[7d:])

VPA in recommendation mode (§27.5) does this calculation for you.

In-place pod resize (beta, on by default since 1.33) changes a running pod's CPU and memory without a restart: kubectl patch pod orders-x --subresource resize -p '{"spec":{"containers":[{"name":"orders","resources":{"requests":{"cpu":"500m"}}}]}}'. It's useful for startup CPU boosts and for VPA's InPlaceOrRecreate mode.

⚠️ Caveat (verify): in-place pod resize was beta (on by default) in 1.33 and may be GA in your version; its API details have changed between releases. The --subresource resize patch needs a matching kubectl (1.32+). Check with kubectl explain pod.spec.containers.resizePolicy. The PromQL in §23.6 uses cAdvisor/kube-state-metrics names; check them in Prometheus before using them.