57. Machine learning in the browser: WebGPU, WebAssembly and Transformers.js

Running a model on the user's device has real advantages. The data never leaves the device, there is no network round trip after the first load, it works offline, and inference costs you nothing per request. The costs are real too: a download of tens to hundreds of megabytes, very different performance from one device to the next, and battery use.

The stack has three layers:

┌──────────────────────────────────────────────────────────────────┐
│ Your JS: pipeline("feature-extraction") / engine.chat.completions │
├──────────────────────────────────────────────────────────────────┤
│ Runtime: ONNX Runtime Web (Transformers.js) · MLC/TVM (WebLLM)    │
│          · TensorFlow.js · MediaPipe                              │
├──────────────────────────────────────────────────────────────────┤
│ Backend: WebGPU (GPU compute shaders)  ·  WebAssembly (CPU, SIMD, │
│          threads via SharedArrayBuffer, §39)  ·  WebNN (emerging)  │
└──────────────────────────────────────────────────────────────────┘

WebGPU from first principles

WebGPU gives JavaScript the GPU as a general-purpose parallel computer, not just a renderer. You write a compute shader in WGSL, give it buffers, and dispatch thousands of threads at once. Every ML runtime in the browser is, underneath, many shaders like this one:

// Add two vectors of 1,000,000 floats on the GPU.
async function gpuAdd(a, b) {
  const adapter = await navigator.gpu?.requestAdapter();
  if (!adapter) throw new Error("WebGPU not available");
  const device = await adapter.requestDevice();

  const shader = device.createShaderModule({
    code: /* wgsl */ `
      @group(0) @binding(0) var<storage, read> a : array<f32>;
      @group(0) @binding(1) var<storage, read> b : array<f32>;
      @group(0) @binding(2) var<storage, read_write> out : array<f32>;

      @compute @workgroup_size(64)
      fn main(@builtin(global_invocation_id) id : vec3u) {
        if (id.x < arrayLength(&out)) {        // the last workgroup may overshoot
          out[id.x] = a[id.x] + b[id.x];
        }
      }`,
  });

  const size = a.byteLength;
  const upload = (data) => {
    const buf = device.createBuffer({ size, usage: GPUBufferUsage.STORAGE, mappedAtCreation: true });
    new Float32Array(buf.getMappedRange()).set(data);
    buf.unmap();
    return buf;
  };
  const bufA = upload(a), bufB = upload(b);
  const bufOut = device.createBuffer({ size, usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC });
  const readback = device.createBuffer({ size, usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ });

  const pipeline = device.createComputePipeline({ layout: "auto", compute: { module: shader, entryPoint: "main" } });
  const bindGroup = device.createBindGroup({
    layout: pipeline.getBindGroupLayout(0),
    entries: [bufA, bufB, bufOut].map((buffer, binding) => ({ binding, resource: { buffer } })),
  });

  const encoder = device.createCommandEncoder();      // record commands…
  const pass = encoder.beginComputePass();
  pass.setPipeline(pipeline);
  pass.setBindGroup(0, bindGroup);
  pass.dispatchWorkgroups(Math.ceil(a.length / 64));  // 15,625 groups × 64 threads
  pass.end();
  encoder.copyBufferToBuffer(bufOut, 0, readback, 0, size);
  device.queue.submit([encoder.finish()]);            // …then submit them all at once

  await readback.mapAsync(GPUMapMode.READ);           // wait for the GPU, asynchronously
  const result = new Float32Array(readback.getMappedRange().slice(0));
  readback.unmap();
  return result;
}

const n = 1_000_000;
const out = await gpuAdd(Float32Array.from({ length: n }, (_, i) => i), new Float32Array(n).fill(0.5));
out[0], out[1], out[n - 1];   // 0.5 1.5 999999.5

This shows the WebGPU model. Work is recorded into command buffers and submitted in one go, because CPU↔GPU synchronization is expensive. Results come back through an async mapAsync. Moving data between CPU and GPU usually costs more than the math, which is why ML runtimes keep tensors on the GPU between layers. WebGPU ships in Chrome/Edge (since 113), in Safari 26 and in Firefox 141 on Windows, with more platforms following. Always feature-detect and fall back to WebAssembly.

Transformers.js: Hugging Face models in a few lines

Transformers.js (@huggingface/transformers, v4 at the time of writing) runs thousands of Hugging Face models (embeddings, classification, translation, speech recognition, small LLMs) on ONNX Runtime Web, using WebGPU or WASM. This is a complete semantic search: it finds documents by meaning, not keywords. The output below is from a real run:

import { pipeline } from "@huggingface/transformers";

// ~23 MB, downloaded once, then served from the browser's Cache Storage.
const embed = await pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2", { device: "webgpu" });

const docs = [
  "How do I reset my password?",
  "Refunds are processed within 5 business days.",
  "Our office is closed on public holidays.",
  "You can change the email address on your account in Settings.",
  "Shipping to Canada takes 7 to 10 days.",
];
// One batched call -> a [5, 384] tensor of unit-length vectors.
const docVecs = (await embed(docs, { pooling: "mean", normalize: true })).tolist();

const dot = (a, b) => a.reduce((s, x, i) => s + x * b[i], 0);   // cosine similarity for unit vectors
async function search(query, k = 2) {
  const [q] = (await embed([query], { pooling: "mean", normalize: true })).tolist();
  return docs.map((text, i) => ({ text, score: dot(q, docVecs[i]) }))
             .sort((a, b) => b.score - a.score).slice(0, k);
}

await search("I forgot my login");
// [{ text: "How do I reset my password?", score: 0.61 }, { text: "You can change the email…", score: 0.32 }]
await search("when do I get my money back?");
// [{ text: "Refunds are processed within 5 business days.", score: 0.59 }, …]

"I forgot my login" shares no words with "reset my password", and it still ranks first. That is what embeddings give you. The same 30 lines power help center search, de-duplicating bug reports, client-side RAG over a user's private notes, and "related articles", all without sending text to a server.

Engineering it properly:

• Run the model in a Worker (§38). The first load parses and compiles the model, and inference on WASM can take hundreds of milliseconds. On the main thread that is an INP disaster (§35). Comlink-style RPC (§41) keeps the call site as await search(q).
• Persist the document vectors in IndexedDB (§45), keyed by a hash of the text, so you only embed new or changed documents.
• Show download progress. pipeline(..., { progress_callback }) reports bytes, and a progress bar turns a 20-second wait into an acceptable first-run experience.
• Pick the backend at runtime. Use device: "webgpu" when navigator.gpu exists, "wasm" otherwise, and quantized weights (dtype: "q8" or "q4") to cut download size.

Real-world example. Google Meet's background blur runs a segmentation model in the browser with WebAssembly SIMD (MediaPipe). Firefox's built-in translation runs its models locally in WASM, so no text leaves the machine. Figma and Photoshop on the web ship large WASM engines. In all of these, privacy and latency were the reasons to do it on the device.