29. Control Theory Basics (Feedback Loops, PID)

Level: Expert You'll use this for: understanding autoscalers, rate limiters, and TCP congestion control as the same underlying pattern.

PID controller — the workhorse of feedback control

output = Kp·error + Ki·∫error dt + Kd·(d(error)/dt)

• P (Proportional): react proportionally to the current error — bigger error, bigger correction.
• I (Integral): react to accumulated past error — corrects persistent small biases (steady-state error) that pure proportional control leaves behind.
• D (Derivative): react to the rate of change of error — dampens oscillation/overshoot by anticipating where the error is heading (this literally reuses the derivative from Ch. 12).

class PIDController:
    def __init__(self, kp, ki, kd, setpoint):
        self.kp, self.ki, self.kd = kp, ki, kd
        self.setpoint = setpoint
        self.integral = 0
        self.prev_error = 0

    def step(self, measured_value, dt=1.0):
        error = self.setpoint - measured_value
        self.integral += error * dt
        derivative = (error - self.prev_error) / dt
        self.prev_error = error
        return self.kp*error + self.ki*self.integral + self.kd*derivative

# Autoscaler: target 60% CPU utilization, adjust replica count
pid = PIDController(kp=0.5, ki=0.1, kd=0.05, setpoint=60)
current_cpu = 85
adjustment = pid.step(current_cpu)
print(adjustment)   # negative -> scale up (reduce per-replica load)

Tracing two PID steps by hand, kp=0.5, ki=0.1, kd=0.05, setpoint=60, starting with integral=0, prev_error=0:

Step 1 — measured CPU = 85:
  error      = setpoint - measured = 60 - 85 = -25
  integral  += error × dt = 0 + (-25×1) = -25
  derivative = (error - prev_error) / dt = (-25 - 0) / 1 = -25
  output = kp·error + ki·integral + kd·derivative
         = 0.5×(-25) + 0.1×(-25) + 0.05×(-25)
         = -12.5   + -2.5        + -1.25
         = -16.25                                <- negative: scale up

Step 2 — CPU eased to 70 (prev_error is now -25):
  error      = 60 - 70 = -10
  integral  += -10  ->  integral = -25 + -10 = -35
  derivative = (-10 - (-25)) / 1 = 15              <- error is shrinking, so this is positive (damping)
  output = 0.5×(-10) + 0.1×(-35) + 0.05×15
         = -5        + -3.5      + 0.75
         = -7.75                                  <- smaller correction: still scaling up, but less aggressively

Notice the integral term keeps accumulating (-25 then -35) even as the error shrinks — that's the "remembers persistent past error" behavior described below — while the derivative term flipped positive the moment the error started improving, pulling the output back toward zero before the P-term alone would.

Why pure proportional control isn't enough (and why over-tuning causes oscillation)

• P-only control typically leaves a persistent steady-state error (it never quite reaches the setpoint, because the correction shrinks to zero as the error shrinks to zero — a vanishing feedback signal, similar in spirit to Ch. 21's vanishing gradient problem).
• Too-aggressive gains (Kp too high) cause overshoot and oscillation — the exact failure mode of an autoscaler that thrashes (scale up, overshoot, scale down, undershoot, repeat) instead of converging smoothly. This is mathematically the same instability pattern as a too-high learning rate in gradient descent (Ch. 21) — both are discrete-time feedback systems that can overshoot and diverge if the step size relative to the system's sensitivity is too large.

TCP congestion control as control theory

Revisiting Ch. 25's AIMD from a control-theory lens: TCP's congestion window is a feedback-controlled variable, where packet loss is the "error signal" telling the sender it exceeded the network's capacity. The conservative asymmetry (slow additive increase, fast multiplicative decrease) is a deliberately stable control law, chosen specifically so that thousands of independent, uncoordinated TCP senders across the global internet converge toward fair bandwidth sharing without a central controller — a remarkable real-world distributed-control-theory result.

Stability, derived: when does a feedback loop converge?

Model the simplest loop in discrete time. Each control tick, a P-controller removes a fraction K of the current error:

e_{t+1} = e_t - K · e_t = (1 - K) · e_t        ->       e_t = (1 - K)ᵗ · e₀

That's a geometric sequence (Ch. 6), so it converges if and only if |1 - K| < 1, that is, 0 < K < 2. It's the identical condition to gradient descent's η < 2/λ in Ch. 21. A training loop and an autoscaler are the same mathematical object. With error e₀ = 10:

For systems with many interacting state variables, the factor becomes a matrix and the rule becomes Ch. 11's: stable if and only if every eigenvalue has magnitude below 1. Control engineers call these eigenvalues the system's poles.

Delay halves the stable gain

Real loops act on stale measurements: metrics scrape intervals, aggregation windows, pod start-up time. Add a delay of just one tick, so the controller corrects based on the error it saw last time:

e_{t+1} = e_t - K · e_{t-1}

Try a solution e_t = zᵗ. Substituting gives the characteristic equation z² - z + K = 0. For K > 1/4 the roots are complex with magnitude √K (the product of the roots is K), so the loop is stable only if K < 1. The stability limit halves from 2 to 1:

The gain that was ideal without delay (K = 1) causes a permanent oscillation with delay. This is the most common cause of autoscaler flapping and of "retry storms" that build in waves: the controller keeps acting on a picture of the world that's already out of date. Longer delays shrink the stable gain further, so the fix is lower gain, a shorter feedback path, or both.

Steady-state error, and why the I-term exists

Suppose a constant disturbance D keeps pushing the system off target, for example a steady background load the autoscaler doesn't see directly. With P-only control the loop settles where the controller's push balances the disturbance:

e_ss = D / (1 + K)

With D = 20 and K = 4, the loop settles with a permanent error of 20/5 = 4. Higher K shrinks that error but, as shown above, moves you toward instability. The integral term Ki · Σ e keeps growing as long as any error remains, so the only place the loop can come to rest is e = 0. That's the mathematical reason the I in PID exists, and also why an unbounded integral causes windup when the actuator saturates (for example, already at max replicas). Production controllers clamp the integral.

The Kubernetes HPA, read as a controller

The Horizontal Pod Autoscaler's core formula is a proportional controller in multiplicative form:

desiredReplicas = ceil( currentReplicas · currentMetric / targetMetric )

Example: 4 replicas running at 90% CPU against a 60% target: ceil(4 · 90/60) = ceil(6) = 6 replicas. Each mechanism the HPA adds around that formula maps onto the maths above:

When an HPA flaps, the diagnosis is the same as for any controller: either the gain is too high (policies too aggressive), the delay is too long (slow metrics, slow pod start-up), or both. KEDA, Karpenter and cloud autoscalers are all variations on this loop.

Real-life engineering ties

• SWE/SRE/Network: autoscalers, rate limiters, adaptive timeout/retry systems, and load-shedding controllers are all PID-flavored (even when implemented as simpler heuristics, they're approximating the same feedback-control idea, often badly, which is why they sometimes oscillate in production).
• Network: TCP congestion control (AIMD, Ch. 25), Wi-Fi's CSMA/CA backoff, and Active Queue Management (CoDel, RED) are all feedback control systems reacting to a congestion signal.
• AI: reinforcement learning control policies for physical/robotic systems build directly on control theory; some learning-rate schedulers are explicitly PID-inspired ("adjust the learning rate based on the rate of change of the loss").
• Security: adaptive rate-limiting against brute-force/DDoS attempts is a feedback controller reacting to a suspicious-traffic "error signal" — poorly tuned gains here cause exactly the oscillation problem above (over-aggressive throttling that also blocks legitimate users in bursts, then relaxes too far, repeating).

More real-life examples:

• Bitcoin's mining difficulty adjusts automatically roughly every two weeks specifically to keep the average block time near 10 minutes regardless of how much total mining power joins or leaves the network — a real, deployed feedback controller operating at global scale with no central operator.
• Cruise control (and adaptive cruise control / lane-keeping in modern cars) is a textbook PID controller: it measures actual speed (or lane position), compares it to the target, and adjusts throttle (or steering) — the same three-term structure as this chapter's code example.
• Data center cooling systems use classic industrial PID controllers to hold server-room temperature at a setpoint despite constantly varying heat load from compute racks — the literal thermostat analogy from this chapter, at industrial scale, and a meaningful lever in a data center's energy efficiency (PUE).
• Kubernetes' Horizontal Pod Autoscaler and most cloud autoscalers are, in effect, simplified proportional (P-only) controllers — exactly why they're prone to the oscillation ("flapping") failure mode described in this chapter, and why production setups add cooldown windows as a crude substitute for proper derivative damping.

Common mistake

Tuning a feedback system (autoscaler, rate limiter) by trial and error in production without understanding why it oscillates — the oscillation is a textbook symptom of gains that are too aggressive relative to the system's response lag, and the fix (add damping via a derivative term, or simply reduce the proportional gain and increase the reaction window) is well understood control theory, not guesswork.