5. API & Endpoint Performance

5.1 Where API latency actually comes from

For a typical CRUD/REST endpoint, budget roughly:

Total p50 = 40ms example breakdown
  Load balancer / ingress:        1-2ms
  App framework routing:          <1ms
  Auth/JWT validation:            1-3ms   (RSA verify is much slower than HMAC)
  Business logic:                 2-5ms
  DB query (indexed):             3-10ms
  DB query (unindexed/full scan): 50-500ms+   <- usual culprit
  Serialization (JSON):           1-5ms   (grows with payload size + nesting)
  Network back to client:         RTT/2

5.2 Payload & serialization costs

JSON is human readable but CPU-costly to (de)serialize at scale; Protocol Buffers / FlatBuffers / MessagePack cut CPU and payload size significantly for high-throughput internal APIs.
Pagination and field selection (GraphQL, sparse fieldsets, ?fields=) prevent "N+1 over-fetch" from blowing up payload size and serialization time under load.
Compression (gzip/br) trades CPU for bandwidth — beneficial on WAN links, harmful on already-CPU-bound backends serving tiny payloads.

5.3 Connection-level tuning that dominates results

Keep-alive: reuse TCP+TLS handshakes (§2.1). Verify your load tool is actually reusing connections (wrk and k6 do by default over HTTP/1.1 keep-alive/HTTP/2; naive scripts using a new requests.get() session per call in Python do not).
Connection pool sizing on both client and server: too small → queueing before a request even starts; too large → downstream resource exhaustion (DB max connections, file descriptor limits).
Timeouts: every hop needs an explicit timeout shorter than its caller's, or a single slow dependency stalls the entire call chain (see §3.2). No timeout = eventual thread-pool/connection-pool exhaustion under any transient slowdown.
Backpressure & load shedding: rate limiting (token bucket/leaky bucket), bulkheads (isolate resource pools per dependency), and circuit breakers prevent a single slow endpoint from cascading. Load tests should explicitly verify these kick in (you should see clean 429/503 responses under overload, not hangs or crashes).

5.4 gRPC / RPC specific concerns

gRPC multiplexes many calls over one HTTP/2 connection — great for efficiency, but means a single misbehaving stream (huge message, slow client reader) can create head-of-line blocking for other RPCs sharing that connection. Load-test gRPC services with multiple connections per client (ghz --connections) to model realistic multi-client concurrency, not one giant shared connection.