77. Advanced techniques: mocks, streams, clocks, users, a11y, visuals

Mock the network to reach states the real server rarely produces

page.route intercepts requests from the page. The agent normally streams a polite reply, so error handling, hostile model output and malformed UI events all need mocks:

// Build an SSE body by hand
export const sse = (...events) => events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join("");

test("a server error shows Retry, and Retry recovers", async ({ chat, page }) => {
  let calls = 0;
  await page.route("**/api/chat", (route) =>
    ++calls === 1
      ? route.fulfill({ status: 500, json: { error: "boom" } })
      : route.fulfill({ contentType: "text/event-stream", body: sse({ type: "text-delta", delta: "Back online." }) }),
  );
  await chat.ask("hello?");
  await expect(page.getByRole("alert")).toContainText("The assistant is unavailable.");
  await page.getByRole("button", { name: "Retry" }).click();
  await expect(chat.lastReply()).toHaveText("Back online.");
  await expect(page.getByRole("alert")).toBeHidden();
});

test("model output is rendered as text, never as HTML", async ({ chat, page }) => {
  const dialogs = [];
  page.on("dialog", (d) => (dialogs.push(d.message()), d.dismiss()));
  const payload = `<img src=x onerror="alert('xss')">`;
  await page.route("**/api/chat", (route) =>
    route.fulfill({ contentType: "text/event-stream", body: sse({ type: "text-delta", delta: payload }) }));
  await chat.ask("hi");
  await expect(chat.lastReply()).toHaveText(payload);           // shown literally…
  await expect(chat.lastReply().locator("img")).toHaveCount(0); // …never parsed
  expect(dialogs).toEqual([]);                                  // and nothing executed
});

test("the UI refuses components it does not know or props that fail validation", async ({ chat, page }) => {
  await page.route("**/api/chat", (route) => route.fulfill({
    contentType: "text/event-stream",
    body: sse(
      { type: "ui", component: "iframe", props: { src: "https://evil.example" } },
      { type: "ui", component: "order-card", props: { id: "X-1", item: "Thing", total: "free", status: "?" } },
      { type: "ui", component: "order-card", props: { id: "X-2", item: "Real thing", total: 5, status: "ok" } },
    ),
  }));
  await chat.ask("show me");
  await chat.waitForReply();
  await expect(chat.lastReply().getByRole("article")).toHaveCount(1);
  await expect(chat.lastReply().getByRole("article")).toHaveAccessibleName("Order X-2");
});

Testing the Stop button needs a request that is still in flight when you click. The route handler holds the response for a second:

test("Stop aborts the stream mid-reply", async ({ chat, page }) => {
  await page.route("**/api/chat", async (route) => {
    const body = sse({ type: "run-started" },
      ...Array.from({ length: 50 }, (_, i) => ({ type: "text-delta", delta: `word${i} ` })),
      { type: "run-finished" });
    await new Promise((r) => setTimeout(r, 1000));
    await route.fulfill({ contentType: "text/event-stream", body }).catch(() => {}); // the page may have aborted
  });
  await chat.ask("tell me a long story");
  await chat.stop.click();
  await expect(chat.lastReply()).toContainText("(stopped)");
  await expect(chat.send).toBeEnabled();
});

Note the .catch(() => {}). The page aborted the request, so fulfilling it fails, and that failure is expected. For WebSocket apps, page.routeWebSocket() mocks the socket in the same style.

API tests in the same suite

The request fixture sends HTTP requests that share the test context's cookies. Security properties are API facts, so check them at the API:

const csrfOf = async (request) => (await (await request.get("/api/me")).json()).csrf;

test("Alice cannot refund Bob's order (no IDOR)", async ({ request }) => {
  const res = await request.post("/api/refunds", {
    data: { orderId: "B-2001" },
    headers: { "x-csrf-token": await csrfOf(request) },
  });
  expect(res.status()).toBe(404); // not 403: don't confirm the order exists
});

test("a cross-site request is refused even with the token", async ({ request }) => {
  const res = await request.post("/api/refunds", {
    data: { orderId: "A-1001" },
    headers: { "x-csrf-token": await csrfOf(request), "sec-fetch-site": "cross-site" },
  });
  expect(res.status()).toBe(403);
});

test("errors never leak stack traces", async ({ request }) => {
  const res = await request.post("/api/chat", {
    headers: { "x-csrf-token": await csrfOf(request), "content-type": "application/json" },
    data: Buffer.from("{not json"), // a string would be JSON-encoded into valid JSON
  });
  expect(res.status()).toBe(400);
  expect(await res.json()).toEqual({ error: "Malformed JSON", requestId: expect.stringMatching(/^[0-9a-f]{12}$/) });
});

That last comment cost a failed run to learn. Playwright JSON-encodes a string passed as data, so "{not json" arrived as the valid JSON string "\"{not json\"". To send raw bytes, pass a Buffer.

Browser facts: cookies

test("the session cookie is HttpOnly and SameSite", async ({ page, context }) => {
  await login(page, USERS.bob);
  await expect(page.getByRole("list", { name: "Your orders" })).toBeVisible();
  const sid = (await context.cookies()).find((c) => c.name === "sid");
  expect(sid).toMatchObject({ httpOnly: true, sameSite: "Lax" });
  expect(await page.evaluate(() => document.cookie)).not.toContain("sid=");   // what XSS would see
});

Two users at once

test("two users in two contexts see only their own data", async ({ browser }) => {
  const [alice, bob] = await Promise.all([browser.newContext(), browser.newContext()]);
  for (const [ctx, user] of [[alice, USERS.alice], [bob, USERS.bob]]) {
    expect((await ctx.request.post("/api/login", { data: user })).ok()).toBe(true);
  }
  const ids = async (ctx) => (await (await ctx.request.get("/api/orders")).json()).map((o) => o.id);
  const [aliceIds, bobIds] = await Promise.all([ids(alice), ids(bob)]);
  // Other tests seed extra orders for Alice, so assert ownership, not exact lists.
  expect(aliceIds).toEqual(expect.arrayContaining(["A-1001", "A-1002"]));
  expect(aliceIds).not.toContain("B-2001");
  expect(bobIds).toEqual(["B-2001"]);
  await Promise.all([alice.close(), bob.close()]);
});

The same pattern tests chat between users, admin/user role differences and "logged out in another tab" flows (with BroadcastChannel, §67, inside one context).

Time: a fake clock instead of waiting 15 minutes

test("an idle session warns before it expires", async ({ page }) => {
  await page.clock.install();          // fake Date, setTimeout, setInterval… BEFORE the app loads
  await page.goto("/");
  await expect(page.getByRole("textbox", { name: "Message" })).toBeVisible();

  await page.clock.fastForward("14:59");
  await expect(page.getByRole("dialog")).toBeHidden();

  await page.clock.fastForward("00:02");
  const dialog = page.getByRole("dialog", { name: "Your session is about to expire" });
  await expect(dialog).toBeVisible();
  await dialog.getByRole("button", { name: "Stay signed in" }).click();
  await expect(dialog).toBeHidden();
});

The test checks both sides of the boundary (not yet at 14:59, warned at 15:01), so a wrong timeout constant fails it. It takes 175 ms.

Accessibility scans

import AxeBuilder from "@axe-core/playwright";

test("the chat screen has no detectable a11y violations @a11y", async ({ chat, page }) => {
  await chat.ask("where is A-1001?");
  await chat.waitForReply();
  const results = await new AxeBuilder({ page }).withTags(["wcag2a", "wcag2aa", "wcag22aa"]).analyze();
  expect(results.violations).toEqual([]);
});

Scan the interesting state, after content has streamed in, not the empty page. Automated scans catch a sizeable minority of accessibility issues: missing names, contrast, invalid ARIA. Keyboard and screen-reader testing by people catches the rest. This scan found a real bug on its first run (§78).

Visual comparison

test("login page looks right @visual", async ({ page }) => {
  await page.goto("/");
  await expect(page.getByRole("heading", { name: "Sign in" })).toBeVisible();
  await expect(page).toHaveScreenshot("login.png");
});

The first run writes the baseline. Later runs compare pixel by pixel, with maxDiffPixelRatio: 0.01 and animations disabled. Fonts render differently on each OS, so baselines are per platform (login-chromium-darwin.png). Either generate CI baselines in the same Docker image CI uses, or skip @visual tests in CI with --grep-invert @visual, as the frontendlabs.xyz site's own CI does. Review every snapshot update in the diff like code.

Passkeys with a virtual authenticator

WebAuthn needs an authenticator, and CI has no fingerprint reader. Chromium's DevTools protocol provides a virtual one. This is the setup used to verify §67's passkey flow:

test("passkey sign-up and sign-in", async ({ page }) => {
  const cdp = await page.context().newCDPSession(page);
  await cdp.send("WebAuthn.enable");
  const { authenticatorId } = await cdp.send("WebAuthn.addVirtualAuthenticator", {
    options: { protocol: "ctap2", transport: "internal", hasResidentKey: true,
               hasUserVerification: true, isUserVerified: true, automaticPresenceSimulation: true },
  });
  // …drive your real registration and sign-in UI here…
  const { credentials } = await cdp.send("WebAuthn.getCredentials", { authenticatorId });
  expect(credentials).toHaveLength(1);
});

CDP sessions are Chromium-only. Mark such tests so the Firefox and WebKit projects skip them.