Week 14 Mini Project: Bot Detection Engine

Build a complete bot detection middleware that integrates all week's signals.

Architecture

Request
    │
    ▼
[JA3Detector — hooks into TLS handshake via GetConfigForClient]
    │
    ▼
[HeaderConsistencyDetector — runs on each HTTP request]
    │
    ▼
[IPReputationDetector — CIDR classification]
    │
    ▼
[BehavioralDetector — session velocity, path diversity, referrer]
    │
    ▼
[BotScorer — aggregates signals → score]
    │
    ▼
[TieredBotMiddleware]
    ├── score 0–20  → allow
    ├── score 20–40 → 500ms delay + allow
    ├── score 40–70 → PoW challenge redirect
    ├── score 70–85 → 403 block
    └── score 85+   → shadow ban (200, empty data)
    │
    ▼
[HoneypotHandler → /api/hidden]
    │ any hit → +100 score, permanent flag
    ▼
[/debug/bots — stats dashboard]

/debug/bots Response Format

{
  "top_scored_ips": [
    {
      "ip": "1.2.3.4",
      "score": 85.0,
      "action": "shadow_ban",
      "signals": [
        {"name": "datacenter_ip", "weight": 15, "reason": "AWS"},
        {"name": "high_velocity", "weight": 20, "reason": "15.2 req/s"},
        {"name": "low_path_diversity", "weight": 10, "reason": "1 unique path in 50 requests"},
        {"name": "ja3_curl", "weight": 25, "reason": "curl TLS fingerprint"},
        {"name": "missing_accept_language", "weight": 10, "reason": "..."}
      ]
    }
  ],
  "honeypot_hits": 3,
  "flagged_ips": 3,
  "challenges_issued": 12,
  "sessions_tracked": 891,
  "action_breakdown": {
    "allow": 8234,
    "delay": 156,
    "challenge": 12,
    "block": 8,
    "shadow_ban": 3
  }
}

Feature Checklist

• Header consistency fingerprinting (UA vs header set)
• JA3-like TLS fingerprint via tls.ClientHelloInfo
• Session behavioral scoring (velocity, path diversity, referrer rate)
• IP ASN classification (datacenter CIDR list for major cloud providers)
• PoW challenge issuance (GET /challenge) and verification (POST /verify)
• Session cookie issued after PoW solved (bypasses future challenges)
• Honeypot endpoint (/api/hidden) — permanent IP flag
• Shadow-ban handler (200 with empty data)
• Tiered mitigation: delay → challenge → block → shadow-ban
• /debug/bots dashboard
• Integration with ThreatIntel store from Phase 3

Engineer Takeaway

DataDome and PerimeterX sell bot detection for $50k–$500k/year. You now understand their core algorithms: header fingerprinting, JA3, behavioral scoring, and tiered mitigation. You can integrate and tune these products with confidence, adjust false-positive thresholds, and judge when in-house detection is enough (internal APIs) and when you need a vendor's breadth (global IP reputation, browser JS challenges).