Part 15 — Architecture decisions by layer: the senior and manager view

15.1 L4 vs L7 load balancing

The gRPC trap: gRPC uses long-lived HTTP/2 connections. Behind an L4 balancer, each client opens one connection to one backend and sends every request there, so new pods get no traffic and hot pods melt. You need per-request L7 balancing (Envoy/service mesh, an L7 LB with HTTP/2 support, or client-side load balancing with a lookaside resolver).

15.2 Where to terminate TLS

Option A: edge only        Client ══TLS══ CDN/LB ──plain── app        simplest; plaintext inside VPC
Option B: re-encrypt       Client ══TLS══ LB ══TLS══ app              compliance-friendly; certs on apps
Option C: pass-through     Client ════════TLS════════ app (L4 LB)     LB can't do L7 routing
Option D: mesh mTLS        Client ══TLS══ LB ══mTLS══ sidecar ─ app   zero-trust; identity per service

Ask: who can see plaintext? Where do certificates live, and who rotates them? What does the compliance scope (PCI, HIPAA) require?

15.3 Ownership map: who owns which layer

Gaps in this table cause incidents. The most common orphaned items: certificate renewal, DNS records, timeout chains across proxies, NAT gateway capacity, and MTU on overlays and VPNs. Assign each one an owner explicitly.

15.4 Design checklist by layer

Use this in design reviews for any new service:

• L1/L2: Which regions/AZs? What RTT do users get? Any L2 assumptions (multicast, VRRP) that won't hold in the cloud? MTU across overlays and VPNs?
• L3: Non-overlapping CIDR from central IPAM? Public vs private subnets? Egress through NAT, with capacity planned? IPv6 plan? Which ICMP is allowed?
• L4: TCP or UDP/QUIC? Connection pooling on every client? Connect/read timeouts set explicitly? Expected connections per second and concurrent connections vs NAT and conntrack limits?
• L5: Stateless servers? Where is session state stored? Idle timeouts documented per hop, each hop shorter than the next? Graceful shutdown and connection draining?
• L6: TLS terminated where, and who can see plaintext? Certificates automated and monitored? Serialisation format and schema-evolution rules? UTF-8, timestamp, and ID conventions?
• L7: Idempotency for writes? Retry policy owned at exactly one layer, with backoff, jitter, and a budget? Rate limiting by what key (not just IP)? Health checks that test real dependencies, but not too aggressively? DNS TTLs for failover?

15.5 Cost lives at layers too

• Data transfer (L3): cross-AZ, cross-region, and Internet egress are billed per GB. A chatty service talking across AZs can cost more than its compute.
• NAT gateway processing (L3/L4): billed per GB. Route traffic to cloud services through VPC endpoints instead.
• L7 features (ALB LCUs, WAF rules, API gateway per-request pricing): grow with requests, not just bytes.