Day 118 — Documentation & Architecture Review
|
|
| Learning objective |
Write an architecture document with ASCII diagrams, design decisions, and operational runbook |
| Security context |
Code you can't explain to your team is a liability; documentation is the knowledge transfer mechanism |
Full Architecture Diagram
┌─────────────────────────────────────────────────────────────┐
│ CLOUD-EDGE SECURITY PROXY (go-proxy) │
│ Single binary, ~5000 LOC │
└─────────────────────────────────────────────────────────────┘
Internet Layer What it does
│
▼
[TCP Listener] L4 Transport TLS termination, mTLS client cert, SNI routing
│ Cert hot-reload (SIGHUP, no drop)
▼
[ProtectedListener] L4 Defense SYN flood: max 5 half-open per IP, 3s drain
│
▼
[SlowlorisDetector] L4 Defense Close conns sending < 50B/s during header phase
│
▼
[JA3 Fingerprinter] Bot/TLS Compute JA3 in GetConfigForClient; flag bad stacks
│
▼
[CIDRRateLimiter] DDoS/L4 Token bucket per /32 + per /24 subnet
│
▼
[TrafficAnalyzer] DDoS EMA z-score: flag IPs > 3σ above baseline
│
▼
[TieredResponder] DDoS Score < 50: allow | 50-80: PoW | >80: block
│
▼
[HTTP Router] L7 net/http, HTTP/1.1 + HTTP/2
│
▼
[SecurityHeaders] OWASP HSTS, CSP, X-Frame-Options, X-Content-Type
│
▼
[RequestLogger] Observability slog JSON, trace context, request ID
│
▼
[HMACAuth/ECDSA] API Auth Middleware chain: try HMAC → ECDSA → mTLS identity
│
▼
[L7RateLimiter] API Per-API-key token bucket (10 req/s, burst 50)
│
▼
[BotScorer] Bot Header consistency + IP reputation + behavior
│
▼
[ChaosMiddleware] Ops Configurable fault injection (disabled by default)
│
▼
[WAFMiddleware] OWASP SQLi + XSS + path traversal + SSRF + body inspect
│
▼
[SchemaValidator] API JSON schema per route, additionalProperties:false
│
▼
[PIIScrubber] Compliance Mask sensitive fields in logs + responses
│
▼
[ReverseProxy] L7 httputil.ReverseProxy, HTTP/2, per-backend CB
│
▼
[ThreatIntel] Aggregation Cross-layer score with exp decay (t½ = 10min)
│ Feeds back into: DDoS tier, bot score, WAF bypass
▼
[Upstreams]
Design Decision Records
## ADR-001: Token Bucket for L7 Rate Limiting
**Context:** Need per-API-key rate limiting at L7.
**Decision:** Token bucket (capacity=50, rate=10/s).
**Alternatives considered:**
- Fixed window: boundary burst issue (user gets 2× limit at window boundary)
- Sliding window log: O(N) memory per key, N = requests in window
- Sliding window counter: O(1), slight approximation
**Rationale:** Token bucket allows legitimate burst (e.g., user refreshes page quickly)
while bounding steady-state rate. Used by AWS API Gateway and Stripe.
**Consequences:** Memory: O(1) per key. Users can burst to 50 requests then are limited to 10/s.
---
## ADR-002: Scoring WAF vs. Rule-Based WAF
**Context:** Should WAF block on first rule match or aggregate scores?
**Decision:** Scoring WAF with per-category thresholds.
**Rationale:** Rule-based WAF has high false positive rate on complex applications
(legitimate search queries contain SQL keywords). Scoring allows tuning sensitivity
per deployment. ModSecurity paranoia levels implement the same concept.
**Consequences:** Some attacks require multiple signals to block. Single-signal attacks
(path traversal) use immediate block regardless of score.
---
## ADR-003: In-Process ThreatIntel vs. External Store
**Context:** Share threat signals between L4 and L7 layers.
**Decision:** In-process sync.Map with exponential decay.
**Alternatives:** Redis (external), SQLite (embedded persistence).
**Rationale:** Single binary, zero external dependencies. Signals are ephemeral
(attack traffic is transient). Memory: ~1KB per IP × 10k IPs = 10MB max.
**Consequences:** Signals lost on restart. Not shared across multiple proxy instances.
Operational Runbook
## Runbook: Blocking an IP
1. Send signal: `kill -HUP $(pgrep go-proxy)`
2. Add to blocklist in config: `echo '{"blocked_cidrs": ["1.2.3.4/32"]}' > /etc/proxy/blocked.json`
3. Reload: `kill -HUP $(pgrep go-proxy)`
4. Verify: `curl http://localhost:8080/debug/vars | jq '.blocked_cidrs'`
## Runbook: Adding a WAF Rule Exception
1. Edit `/etc/proxy/waf-exceptions.json`
2. Add: `{"path": "/api/search", "skip_rules": ["sqli"]}`
3. Reload: `kill -HUP $(pgrep go-proxy)`
4. Monitor: `tail -f /var/log/proxy/waf-audit.log | jq 'select(.action=="allow")'`
## Runbook: Emergency Config Rollback
1. Stop the rollout: `kill -TERM $(pgrep go-proxy)` — graceful drain, 30s
2. Start old version: `systemctl start go-proxy@previous`
3. Verify health: `curl http://localhost:8080/healthz`