15. Specialized Indexes: Search, Geo, Time-Series

15.1 Full-text search

Inverted index: term → posting list of (doc_id, positions, frequency).

Doc1: "red running shoes"     Doc2: "blue running jacket"
running → [1, 2]   red → [1]   shoes → [1]   blue → [2]   jacket → [2]
Query "running shoes" → intersect posting lists → [1]; rank by BM25 / learned ranking

Architecture: the source-of-truth DB → CDC/stream → indexer → search cluster (sharded by doc, replicated). Queries go scatter-gather across shards → merge the top-K. The search index is never the source of truth, because it can always be rebuilt.

Modern ranking is a multi-stage funnel: retrieval (BM25 and/or vector ANN, which returns ~1000 candidates) → lightweight ranker (~100) → heavy ML re-ranker (~10).

15.2 Geospatial

15.3 Time-series

Patterns: append-only writes, queries over recent time ranges, downsampling (raw for 7 days → 1-minute rollups for 90 days → 1-hour rollups for years), delta/XOR compression (Facebook's Gorilla paper), and a partition key of (metric, time_bucket). Cardinality (the number of unique label combinations) is the cost killer. Never put user IDs in metric labels.

15.4 🏗️ Architecture: search indexing and query paths

 Indexing                                              Query
 ────────                                              ─────
 Source DB ─CDC─▶ Kafka ─▶ Indexer workers              User ─▶ Search API
                   (enrich, tokenize, embed)                   │ parse, spell-correct, rewrite
                         │ bulk index                          ▼
                         ▼                             Coordinator ─scatter─▶ shard 1..N (each: replica pick)
             ┌──── Index (alias "products") ───┐                 ◀─gather top-K each─
             │ shard1  shard2  ...  shardN     │      merge → re-rank (ML) → hydrate from DB/cache
             └─────────────────────────────────┘      → apply ACL/business filters → respond
 Full reindex: build "products_v8" in parallel → backfill → catch up from Kafka
               → atomically swap alias products → v8 → keep v7 for rollback

Geo query flow (nearby drivers within 2 km):

 pickup (lat,lng) → H3 cell at res 8 → k-ring(k=2) = 19 cells → look up drivers per cell
 (in-memory index sharded by cell) → filter by exact haversine distance & status → top N by ETA

15.5 Deeper: internals

• Lucene segments: new docs go to an in-memory buffer and are refreshed into immutable segments, by default every ~1 s in Elasticsearch. Search is therefore near real time, not immediate. Segments are merged in the background. Deletes are tombstones until a merge.
• Relevance across shards: term statistics (IDF) are per shard, so the same doc can score differently depending on its shard. Use DFS query-then-fetch or enough docs per shard to smooth the statistics.
• Deep pagination: from=100000 makes every shard return 100,010 docs to the coordinator. Use search_after cursors, or a point-in-time snapshot.
• Geohash edge effects: nearby points can have completely different prefixes across a cell boundary, or across the equator, prime meridian, or antimeridian. Always query the neighboring cells.
• Time-series compression (Facebook Gorilla, 2015): delta-of-delta timestamps and XOR-encoded float values compress to roughly 1.37 bytes per data point, which made in-memory storage of 26 hours of metrics feasible.

15.6 🌍 In production

• Uber H3 (open-sourced 2018): hexagonal hierarchical grid used for surge pricing, supply/demand forecasting, and dispatch. Hexagons have equal distances to all 6 neighbors, unlike squares, which have 4 edge neighbors and 4 corner neighbors.
• Google S2: spherical geometry on Hilbert-curve-ordered cells, used in Google Maps-scale systems and by Niantic for Pokémon GO.
• Twitter Earlybird (2012): a real-time search engine on modified Lucene, designed so tweets are searchable within seconds.
• GitHub code search (2023, "Blackbird"): they moved off Elasticsearch to a custom Rust engine built on n-gram indexes and content-addressed dedupe of identical files across forks, because general-purpose full-text search didn't fit code at their scale.
• Uber M3 and Facebook Gorilla/Beringei: purpose-built time-series stores, because general databases couldn't handle the write rate and cardinality.

15.7 ⚠️ Corner cases & deep details

• Index and DB divergence: a missed CDC event or a failed bulk request leaves the index out of sync. Run periodic reconciliation (compare counts and checksums per partition), and always be able to rebuild from source.
• Mapping explosion: indexing arbitrary user JSON creates thousands of fields, which exhausts cluster state memory. Use explicit mappings and flattened types.
• Visibility and ACLs: a document is indexed before its permissions propagate, and it leaks in search results. Index ACLs with the document and filter at query time, or hide it until it's fully processed.
• Moving objects: drivers move between cells every few seconds. Updating geo indexes with delete-and-insert at 1M updates/s is heavy, so keep it in memory with per-cell sets and accept a few seconds of staleness.
• Density variance: fixed-resolution cells are crowded downtown and empty in rural areas. Use adaptive resolution (a quadtree, or several H3 resolutions): expand the search ring until N candidates are found.
• Metrics cardinality explosion: one engineer adds user_id as a label, creating 100M time series and taking down the metrics cluster (or the bill). Enforce cardinality budgets at ingestion.
• Counter resets: a process restart resets a counter to 0. Rate calculations must detect resets (Prometheus rate() does).