4. Load & Traffic Modeling
4.1 Traffic shapes seen in production
| Pattern | Description | Real example |
|---|---|---|
| Diurnal | Smooth daily sine wave | B2B SaaS, business-hours traffic |
| Flash spike | 10-100x traffic in seconds | Flash sale, celebrity tweet, news event |
| Thundering herd | Many clients retry simultaneously after an outage or cache expiry | Cache stampede, mass token refresh at midnight |
| Sawtooth | Steady growth then hard reset | Batch jobs, cron-triggered fan-out |
| Poisson / random arrivals | Independent, memoryless arrivals | General public web traffic baseline |
| Bursty / self-similar | Traffic with long-range dependence, not smooth Poisson | CDN request logs, video streaming (real internet traffic is provably NOT Poisson — see Leland et al.'s "self-similar nature of Ethernet traffic") |
4.2 Modeling arrival rates with Poisson / exponential inter-arrival
For a target rate λ req/s, inter-arrival times between requests should be drawn from
an exponential distribution (Exp(λ)) if you want to simulate independent random
users, not a fixed period (fixed period = "closed-loop" bursting in lockstep, which
under-stresses connection-acceptance queues).
import random
def next_interarrival_seconds(rate_per_sec: float) -> float:
return random.expovariate(rate_per_sec)
4.3 Ramp patterns for finding breaking points
- Step load: hold at 100 req/s for 1 min, then 200, then 400... until errors/ latency degrade. Cheap and fast; identifies the rough ceiling.
- Ramp (soak-in): linear increase over 10-30 min. Reveals gradual resource leaks (connection leaks, memory growth, GC pressure) that step load's short plateaus might miss.
- Soak test: fixed moderate load for hours. Catches memory leaks, log-disk fill-up, certificate/token expiry bugs, slow degradation from fragmentation.
- Spike test: baseline load, then a sudden 10x spike for 30-60s, then back to baseline. Tests autoscaling reaction time and overload protections (rate limiting, circuit breakers, load shedding).
- Breakpoint / stress test: increase until the system actually fails, to learn how it fails (graceful 503s + shed load? Or does it fall over and take dependencies with it — e.g., DB connection storm during retries?).