Week 7 Mini Project: Production L4 Load Balancer

Full Architecture

                        ┌─────────────────────────┐
Clients ──TCP──▶       │   L4 Load Balancer :8080  │
                        │                           │
                        │  ┌──────────────────────┐ │
                        │  │ mTLS Termination      │ │
                        │  └──────────────────────┘ │
                        │  ┌──────────────────────┐ │
                        │  │ Per-IP Rate Limiter   │ │
                        │  └──────────────────────┘ │
                        │  ┌──────────────────────┐ │
                        │  │ Least-Conn Balancer   │ │
                        │  └──────────────────────┘ │
                        │  ┌──────────────────────┐ │
                        │  │ Circuit Breaker/Pool  │ │
                        │  └──────────────────────┘ │
                        │                           │
                        │  Observability: :9090     │
                        │  /stats /debug/vars       │
                        └─────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              Backend :8081   Backend :8082   Backend :8083

Features Checklist

Least-connections balancing (switchable to round-robin)
Active health checking (TCP probe every 5s)
Connection draining (graceful backend removal)
Per-IP connection rate limiting (100/IP, 1000 global)
Circuit breaker per backend (trips at 5 failures, recovers after 30s)
Connection pool to backends (max 20 per backend)
Full metrics at /debug/vars and /stats
Graceful shutdown with drain

Test the circuit breaker:

# Kill backend :8082 while serving traffic
# Watch the circuit open, then test recovery when you restart it
curl http://localhost:9090/stats

Engineer Takeaway

AWS NLB handles 100 million connections/day. You've now built the core algorithms that power it. The pricing model (per LCU = per connection + per bandwidth) makes sense now because you've implemented the primitives. When you configure a managed load balancer, you'll know what each setting does underneath. And when someone proposes building one in-house, you can explain why the answer is almost always "use the managed one", and when it isn't.