Chapter 68 — Project 5: distill a tiny model and run it in the browser (Bengaluru, next 30 days)

Step 1 — Two data sources for the teacher: normals, plus a live feed

Bengaluru's long-term monthly averages (approximate, illustrative) anchor a smooth seasonal curve:

# Approximate long-term monthly climate normals for Bengaluru, India.
# (day-of-year at mid-month, avg high C, avg low C, avg humidity %, avg MONTHLY rainfall mm)
MONTHLY = [
    (15,  27, 15, 55,   5),   # mid-Jan  -- cool, dry
    (46,  30, 17, 45,  10),   # mid-Feb
    (74,  33, 19, 40,  15),   # mid-Mar  -- warming fast
    (105, 34, 21, 50,  45),   # mid-Apr  -- hottest
    (135, 33, 20, 60, 120),   # mid-May  -- pre-monsoon showers
    (166, 29, 19, 70,  80),   # mid-Jun  -- SW monsoon onset
    (196, 27, 19, 75, 110),   # mid-Jul
    (227, 27, 18, 75, 140),   # mid-Aug
    (258, 28, 18, 73, 170),   # mid-Sep  -- wettest
    (288, 27, 18, 75, 150),   # mid-Oct  -- NE monsoon
    (319, 26, 17, 70,  60),   # mid-Nov  -- retreating monsoon
    (349, 26, 15, 60,  15),   # mid-Dec  -- cool, dry
]

Cyclic linear interpolation between consecutive months turns these 12 anchor points into a smooth value for any day of the year (climatology()). Rainfall is stored as a monthly total but converted to an average daily figure at interpolation time (/ 30.44) — a small but important unit fix, so it's directly comparable to real daily rainfall from the second source.

The second source is a live feed. fetch_live_history() pulls 3 years of real daily history for Bengaluru (12.9716°N, 77.5946°E) from Open-Meteo's free historical-weather API — no key required, stdlib urllib only:

def fetch_live_history(years=3, timeout=15):
    end = date.today() - timedelta(days=5)
    start = end - timedelta(days=365 * years)
    url = (
        "https://archive-api.open-meteo.com/v1/archive"
        f"?latitude={LAT}&longitude={LON}"
        f"&start_date={start.isoformat()}&end_date={end.isoformat()}"
        f"&daily={LIVE_FIELDS}&timezone=Asia%2FKolkata"
    )
    try:
        with urllib.request.urlopen(url, timeout=timeout) as resp:
            payload = json.loads(resp.read().decode())
    except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
        print(f"[warn] Live feed unavailable ({e}); training on illustrative "
              f"climatology only.")
        return []
    # ... parse payload["daily"] into (day_of_year, [tmax, tmin, hum, rain_mm], date) tuples

The failure mode is handled, not assumed away. If there's no network, the except block logs a warning and returns an empty list — the rest of the script treats "zero live samples" as a valid state and trains on the illustrative curve alone. Nothing about this project requires internet access; it's just better with it.

When the fetch succeeds, the script prints a genuine sanity check — the hand-typed table against what actually happened:

Month hand tmax/tmin/hum    live-observed avg
1     27/15/55%             27.4/16.5/64% rain=0.2mm/day
5     33/20/60%             31.5/21.6/69% rain=4.3mm/day
9     28/18/73%             28.0/19.5/79% rain=3.3mm/day

Close, but not exact — the real data runs a few points more humid across the board than the hand-typed guesses. This is the kind of thing you only catch by actually checking against a live source, which is precisely the point of adding one.

Each day is represented as 4 seasonal features, not a raw day number — sine and cosine at the annual and semiannual frequency, so the network sees a smooth, cyclic (Dec 31 next to Jan 1) representation instead of an arbitrary integer:

def features(doy):
    ang = 2 * math.pi * doy / 365.0
    return [math.sin(ang), math.cos(ang), math.sin(2*ang), math.cos(2*ang)]

Step 2 — Train the teacher on the blend (the "big, accurate" model)

A 4 -> 24 -> 24 -> 4 MLP (tanh hidden layers, linear output — same shape of network as Chapter 14, just deeper). Each epoch, its training batch is the union of noisy synthetic samples from climatology() and real samples from the live feed — real observations are used as-is, with no synthetic noise added, since they already carry real-world noise:

teacher = MLP([4, 24, 24, 4])
for epoch in range(150):
    batch = []
    for d in days:
        noisy = [v + random.gauss(0, s * 0.08) for v, s in zip(raw_targets[d], std)]
        batch.append((features(d), normalize(noisy)))
    for doy, t, _ in live_train:
        batch.append((features(doy), normalize(t)))
    random.shuffle(batch)
    for x, y in batch:
        teacher.train_step(x, y, lr=0.05)

(MLP is a plain-Python class with manual forward/backward — no numpy or torch needed; see the full listing in train_and_distill.py, built the same way Chapter 15 built backpropagation from scratch.) This teacher has 820 parameters. The most recent 60 days of the live feed are held out of live_train entirely — kept aside as a validation set the teacher (and later the student) never sees during training, so Step 4's accuracy numbers are honest, not graded on the training set (Chapter 13's golden rule).

Step 3 — Distill the student (the one that goes in the browser)

A 4 -> 8 -> 4 MLP — 76 parameters, about 1/11th the size. Following Chapter 67's regression-distillation recipe exactly: the student never sees raw_targets, the live feed, the noise, or a single ground-truth label. Its only training signal, ever, is the teacher's own output:

student = MLP([4, 8, 4])
for epoch in range(2500):
    random.shuffle(days)
    for d in days:
        x = features(d)
        teacher_out = teacher.forward(x)[-1]   # <-- the only label the student ever sees
        student.train_step(x, teacher_out, lr=0.08)

Blending in a live feed changed what the teacher learned to reproduce; it changed nothing about how the student learns to copy the teacher — that's the value of separating these two concerns into two steps.

Step 4 — What actually happened (real numbers from this run)

Teacher params: 820
Student params: 76
Compression ratio: 10.8x

Teacher vs synthetic curve, mean MSE (physical units^2): 16.5304
Student vs synthetic curve, mean MSE (physical units^2): 16.6014
Student vs teacher,        mean MSE (normalized units^2): 0.000821

HELD-OUT VALIDATION -- 60 real days neither model trained on:
Teacher vs real held-out days, mean MSE: 29.4713
Student vs real held-out days, mean MSE: 29.2699

Three things worth stopping on:

1. The "vs synthetic curve" MSE is much higher than the climatology-only version of this project (16.5 vs. an earlier run's 2.0). That's not a regression — the teacher is no longer purely fitting the smooth hand-typed curve; it's compromising between that curve and real, noisier observations that don't sit exactly on it. A model that fits both sources reasonably well necessarily fits either one alone somewhat worse than a model that only ever saw that one source.
2. The student still matches the teacher almost exactly (MSE 0.0008 in normalized units, up slightly from 0.00005 because the function it's copying is now a little less smooth) — an 76-parameter network still reproduces an 820-parameter network's function to within a rounding error, even though that function is now shaped by real data, not a hand- typed curve alone.
3. On the honest test — 60 real days neither model trained on — the 76-parameter student (MSE 29.27) is again fractionally more accurate than its 820-parameter teacher (MSE 29.47). This is the same effect Chapter 67 predicted and the original version of this chapter measured on the synthetic curve — now confirmed on genuine held-out weather, not just an idealized function. The teacher absorbs a little noise from what it's trained on; the student, trained on the teacher's smoothed output, doesn't inherit all of it.

Step 5 — Export: the entire model is a <script> tag

export = {
    "arch": [4, 8, 4],
    "outputs": ["temp_max_C", "temp_min_C", "humidity_pct", "rain_mm_per_day"],
    "trained_with_live_data": bool(live_train),
    "live_data_range": [live_all[0][2], live_all[-1][2]] if live_all else None,
    "mean": mean, "std": std,        # to denormalize outputs back to °C/%/mm
    "W1": student.W[0], "b1": student.b[0],
    "W2": student.W[1], "b2": student.b[1],
}
with open("weights.js", "w") as f:
    f.write("const WEATHER_MODEL = " + json.dumps(export, indent=2) + ";\n")

Result: weights.js, containing every number the browser will ever need, plus a little provenance metadata (trained_with_live_data, live_data_range) so the app itself can honestly disclose what it was trained on. Compare 76 floats to the 14 GB in Chapter 66's table for a 7B LLM at bf16 — nine orders of magnitude apart, same underlying idea (a file full of weights that a runtime loads and multiplies).

Note what's not in this file: the teacher's weights, the raw live-feed rows, and the hand-typed MONTHLY table are all absent. The browser gets the distilled behaviour, not the data or the process that produced it.

Step 6 — The browser runtime: no TensorFlow.js, no ONNX Runtime

At 76 parameters, you don't need a model-serving library — you need a matrix-vector multiply, by hand, exactly like Chapter 24 built attention by hand:

function forward(x) {
  const { W1, b1, W2, b2, mean, std } = WEATHER_MODEL;
  const h = W1.map((row, i) =>
    Math.tanh(row.reduce((s, w, k) => s + w * x[k], 0) + b1[i]));
  const outNorm = W2.map((row, i) =>
    row.reduce((s, w, k) => s + w * h[k], 0) + b2[i]);
  return outNorm.map((v, i) => v * std[i] + mean[i]);  // denormalize
}

function featuresForDate(date) {
  const start = Date.UTC(date.getUTCFullYear(), 0, 1);
  const doy = Math.floor((date - start) / 86400000);
  const ang = 2 * Math.PI * doy / 365;
  return [Math.sin(ang), Math.cos(ang), Math.sin(2 * ang), Math.cos(2 * ang)];
}

function predictDay(date) {
  const [tempMax, tempMin, humidity, rainMm] = forward(featuresForDate(date));
  return { date, tempMax, tempMin, humidity, rainMm: Math.max(0, rainMm) };
}

That's the whole "inference engine" — about ten lines, no dependency, and it runs in any browser, offline, after the page has loaded once. index.html wraps this in a loop over the next 30 days, a canvas line chart, and a table (full listing in the project folder), and reads WEATHER_MODEL.live_data_range to show what real-world data window the model was trained on. Every prediction on the page is computed live, client-side, from weights.js — there is no network call at forecast time, only at training time.

A real, unfixed rough edge: rainMm is an unconstrained linear regression output, so on very dry days it can predict a small negative number of millimetres — physically impossible. Math.max(0, rainMm) clips it at display time. The honest fix belongs in the model (a non-negative output activation, or a zero-inflated formulation), not in the UI layer — left as a rough edge deliberately, because pretending it isn't there would be less useful to you than seeing it.

Step 7 — Run it

cd projects/bangalore-weather-distilled
python3 train_and_distill.py   # optional -- fetches live data if you have network
open index.html                # macOS: opens directly in your default browser

No npm install, no dev server, no build step — this is the point of distilling down to a model this small: the deployment story collapses to "put a file on a web page."

Experiments (do at least two)

1. Shrink the student further. Change [4, 8, 4] to [4, 3, 4] and retrain. Watch the held-out MSE rise — you're finding the actual compression cliff for this function, not guessing at it.
2. Distill without any labelled data at all. Confirm the student's training loop (Step 3) really does compile and train with raw_targets, live_train, and climatology() deleted from that loop entirely — proof that response-based distillation genuinely needs no labelled data, only a trained teacher.
3. Add an uncertainty band. Train 5 students with different random seeds and plot their spread alongside the mean — a cheap way to visualise "how confident is this pattern, really" for a browser app with no server-side inference cost.
4. Fix the negative-rainfall rough edge properly. Replace the student's linear output for the rain dimension with a softplus (log(1 + e^x), always positive) and retrain. Confirm the held-out MSE on rain doesn't get worse — a non-negative output shouldn't cost you accuracy, only remove an impossible value.
5. Widen the live feed. Increase LIVE_YEARS from 3 to 10 and compare the held-out MSE — more real data should tighten the teacher's fit without changing anything about how the student is distilled from it.