Edge Case Testing. Find Them Without Listing Them

-- Sentry / PostHog error grouping query for edge case test backlog.
--
-- Goal: group production error events by the dimensions that matter
-- for reproduction in a test (URL path, client/browser context, error
-- type) and rank the groups by occurrence count. The top groups are
-- your Tier 1 edge case test candidates.
--
-- ============================================================
-- Column name swaps for Sentry vs PostHog
-- ============================================================
-- This query is written against the PostHog `events` table where:
--   event              = '$exception'  (PostHog exception event)
--   properties.$pathname            -> URL path of the error
--   properties.$browser              -> browser/client name
--   properties.$exception_type       -> error class (e.g. TypeError)
--   properties.$exception_message    -> error message
--
-- For a Sentry export (e.g. via Sentry's CSV/Snowflake/BigQuery
-- integration), swap the columns as follows:
--   PostHog                          Sentry export
--   ---------------------------------------------------
--   properties.$pathname          -> transaction        (or url)
--   properties.$browser           -> contexts.browser.name
--   properties.$exception_type    -> type
--   properties.$exception_message -> message            (or title)
--   timestamp                     -> received           (or timestamp)
--   event = '$exception'          -> level = 'error'    (filter)
--
-- The aggregation shape stays identical; only the column references
-- change. Adapt the FROM clause to the table that holds your error
-- events in whichever warehouse you have configured.

SELECT
    properties.$pathname            AS url_path,
    properties.$browser             AS browser,
    properties.$exception_type      AS error_type,
    properties.$exception_message   AS error_message,
    COUNT(*)                        AS occurrence_count,
    COUNT(DISTINCT distinct_id)     AS affected_users,
    MIN(timestamp)                  AS first_seen,
    MAX(timestamp)                  AS last_seen
FROM events
WHERE event = '$exception'
  AND timestamp >= NOW() - INTERVAL 30 DAY
GROUP BY
    url_path,
    browser,
    error_type,
    error_message
ORDER BY occurrence_count DESC
LIMIT 100;

/**
 * Playwright boundary-value test for a numeric quantity input.
 *
 * The field under test is expected to:
 *   - accept the minimum valid value (1)
 *   - accept the maximum valid value (99)
 *   - reject values above the maximum (100) with a visible error
 *   - reject zero with a visible error
 *
 * The assertion goes one step beyond "the input accepts the value".
 * For rejected inputs we also assert that the server endpoint never
 * receives the out-of-range value, by listening for the POST request
 * and failing the test if it fires with a bad payload.
 *
 * Replace BASE_URL and the selectors with your application's values.
 */

import { test, expect, type Request } from "@playwright/test";

const BASE_URL = process.env.BASE_URL ?? "http://localhost:3000";
const FORM_PATH = "/cart";
const SUBMIT_ENDPOINT = "/api/cart/update";

const QUANTITY_INPUT = '[data-testid="quantity-input"]';
const SUBMIT_BUTTON = '[data-testid="submit-quantity"]';
const ERROR_MESSAGE = '[data-testid="quantity-error"]';
const SUCCESS_MESSAGE = '[data-testid="quantity-success"]';

async function fillAndSubmit(page: import("@playwright/test").Page, value: string) {
    await page.goto(`${BASE_URL}${FORM_PATH}`);
    await page.locator(QUANTITY_INPUT).fill(value);
    await page.locator(SUBMIT_BUTTON).click();
}

function captureUpdateRequests(page: import("@playwright/test").Page): Request[] {
    const captured: Request[] = [];
    page.on("request", (req) => {
        if (req.url().includes(SUBMIT_ENDPOINT) && req.method() === "POST") {
            captured.push(req);
        }
    });
    return captured;
}

test.describe("quantity input boundary values", () => {
    test("accepts the minimum valid value (1)", async ({ page }) => {
        const requests = captureUpdateRequests(page);
        await fillAndSubmit(page, "1");
        await expect(page.locator(SUCCESS_MESSAGE)).toBeVisible();
        await expect(page.locator(ERROR_MESSAGE)).toHaveCount(0);
        expect(requests.length).toBe(1);
    });

    test("accepts the maximum valid value (99)", async ({ page }) => {
        const requests = captureUpdateRequests(page);
        await fillAndSubmit(page, "99");
        await expect(page.locator(SUCCESS_MESSAGE)).toBeVisible();
        await expect(page.locator(ERROR_MESSAGE)).toHaveCount(0);
        expect(requests.length).toBe(1);
    });

    test("rejects max+1 (100) with a visible error and never POSTs the bad value", async ({ page }) => {
        const requests = captureUpdateRequests(page);
        await fillAndSubmit(page, "100");
        await expect(page.locator(ERROR_MESSAGE)).toBeVisible();
        await expect(page.locator(ERROR_MESSAGE)).toContainText(/maximum|too\s*(high|large)|99/i);
        const badPayloads = requests.filter((r) => {
            const body = r.postData() ?? "";
            return body.includes("\"quantity\":100") || body.includes("quantity=100");
        });
        expect(badPayloads.length).toBe(0);
    });

    test("rejects zero with a visible error and never POSTs the bad value", async ({ page }) => {
        const requests = captureUpdateRequests(page);
        await fillAndSubmit(page, "0");
        await expect(page.locator(ERROR_MESSAGE)).toBeVisible();
        await expect(page.locator(ERROR_MESSAGE)).toContainText(/at least|minimum|greater than 0|1/i);
        const badPayloads = requests.filter((r) => {
            const body = r.postData() ?? "";
            return body.includes("\"quantity\":0") || body.includes("quantity=0");
        });
        expect(badPayloads.length).toBe(0);
    });
});

/**
 * Vitest unit tests for string-boundary conditions on a hypothetical
 * user-profile `bio` field handler.
 *
 * Behavior under test:
 *   - empty string input  -> returns empty string (or null), never throws
 *   - whitespace-only     -> trimmed to empty string (or null)
 *   - 10,000-char input   -> truncated to documented MAX_BIO_LENGTH,
 *                            never silently passed through
 *   - valid input         -> returned as-is (trimmed)
 *
 * If your real handler returns `null` instead of `""` for the empty
 * case, set EMPTY_BIO_RETURN below to null and the tests stay green.
 */

import { describe, it, expect } from "vitest";

export const MAX_BIO_LENGTH = 280;
const EMPTY_BIO_RETURN: string | null = "";

export function normalizeBio(input: unknown): string | null {
    if (input === undefined || input === null) {
        return EMPTY_BIO_RETURN;
    }
    if (typeof input !== "string") {
        throw new TypeError(`bio must be a string, got ${typeof input}`);
    }
    const trimmed = input.trim();
    if (trimmed.length === 0) {
        return EMPTY_BIO_RETURN;
    }
    if (trimmed.length > MAX_BIO_LENGTH) {
        return trimmed.slice(0, MAX_BIO_LENGTH);
    }
    return trimmed;
}

describe("normalizeBio - string boundary conditions", () => {
    it("returns the empty-bio sentinel for an empty string (does not throw)", () => {
        expect(() => normalizeBio("")).not.toThrow();
        expect(normalizeBio("")).toBe(EMPTY_BIO_RETURN);
    });

    it("returns the empty-bio sentinel for whitespace-only input (trim behavior)", () => {
        expect(normalizeBio("   ")).toBe(EMPTY_BIO_RETURN);
        expect(normalizeBio("\t\n  \n")).toBe(EMPTY_BIO_RETURN);
    });

    it("truncates a 10,000-character input to MAX_BIO_LENGTH (no silent passthrough)", () => {
        const longInput = "a".repeat(10_000);
        const result = normalizeBio(longInput);
        expect(typeof result).toBe("string");
        expect((result as string).length).toBe(MAX_BIO_LENGTH);
        expect((result as string).length).toBeLessThan(longInput.length);
    });

    it("returns a normal-length bio unchanged (trimmed)", () => {
        expect(normalizeBio("  hello world  ")).toBe("hello world");
    });

    it("handles null and undefined without throwing", () => {
        expect(() => normalizeBio(null)).not.toThrow();
        expect(() => normalizeBio(undefined)).not.toThrow();
        expect(normalizeBio(null)).toBe(EMPTY_BIO_RETURN);
        expect(normalizeBio(undefined)).toBe(EMPTY_BIO_RETURN);
    });
});

/**
 * Playwright file-upload round-trip test.
 *
 * Reproduces the bug where a filename containing parentheses and a
 * space is URL-encoded at upload time but the download endpoint
 * returns the percent-encoded form in Content-Disposition, so the
 * filename the user sees on download does not match the filename
 * they uploaded.
 *
 * The test:
 *   1. Creates a temp file named "report (2024) final.pdf".
 *   2. Uploads it via the file input.
 *   3. Reads the download endpoint's Content-Disposition header.
 *   4. Asserts the filename in the header equals the original,
 *      with NO percent-encoding (no "%20", no "%28", no "%29").
 *
 * Replace BASE_URL and selectors with your application's values.
 */

import { test, expect } from "@playwright/test";
import * as fs from "node:fs";
import * as path from "node:path";
import * as os from "node:os";

const BASE_URL = process.env.BASE_URL ?? "http://localhost:3000";
const UPLOAD_PATH = "/uploads/new";
const FILE_INPUT = 'input[type="file"]';
const SUBMIT_BUTTON = '[data-testid="upload-submit"]';
const UPLOADED_LINK = '[data-testid="last-upload-link"]';

const TRICKY_FILENAME = "report (2024) final.pdf";

test("uploaded filename round-trips through download without percent-encoding", async ({ page, request }) => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "edge-case-upload-"));
    const filePath = path.join(tmpDir, TRICKY_FILENAME);
    fs.writeFileSync(filePath, "%PDF-1.4\nplaceholder body\n");

    try {
        await page.goto(`${BASE_URL}${UPLOAD_PATH}`);
        await page.locator(FILE_INPUT).setInputFiles(filePath);
        await page.locator(SUBMIT_BUTTON).click();

        const link = page.locator(UPLOADED_LINK);
        await expect(link).toBeVisible();
        const downloadUrl = await link.getAttribute("href");
        expect(downloadUrl, "download link must exist").toBeTruthy();

        const absoluteUrl = downloadUrl!.startsWith("http")
            ? downloadUrl!
            : `${BASE_URL}${downloadUrl!}`;
        const response = await request.get(absoluteUrl);
        expect(response.status(), "download endpoint should return 200").toBe(200);

        const disposition = response.headers()["content-disposition"];
        expect(disposition, "Content-Disposition header must be present").toBeTruthy();

        expect(disposition).not.toMatch(/%20/);
        expect(disposition).not.toMatch(/%28/);
        expect(disposition).not.toMatch(/%29/);

        const filenameMatch = disposition!.match(/filename\*?=(?:UTF-8'')?["]?([^";]+)["]?/i);
        expect(filenameMatch, `could not parse filename from: ${disposition}`).toBeTruthy();
        const returnedFilename = decodeURIComponent(filenameMatch![1]);
        expect(returnedFilename).toBe(TRICKY_FILENAME);
    } finally {
        try {
            fs.rmSync(tmpDir, { recursive: true, force: true });
        } catch {
            // best-effort cleanup
        }
    }
});

/**
 * Vitest two-part locale-coverage test.
 *
 * Part 1 - JSON validity:
 *   Walk every file matching `locales/*.json`, parse it, and fail the
 *   test if any file is not valid JSON. Malformed JSON in a locale
 *   bundle ships as a runtime crash in the browser; catch it in CI.
 *
 * Part 2 - Key coverage:
 *   Treat `locales/en.json` as the canonical key set. For every other
 *   locale file, assert that every key (recursively, dot-paths) that
 *   exists in `en.json` also exists in the other locale. The bug this
 *   catches: a key added to `en.json` but forgotten in `fr.json`
 *   renders the raw key string ("checkout.shipping.estimate") to
 *   French-locale users.
 *
 * The test resolves locale files relative to the project root via a
 * configurable LOCALES_DIR. Default points at `locales/` next to the
 * project root.
 */

import { describe, it, expect, beforeAll } from "vitest";
import * as fs from "node:fs";
import * as path from "node:path";
import { globSync } from "glob";

const LOCALES_DIR = process.env.LOCALES_DIR
    ? path.resolve(process.env.LOCALES_DIR)
    : path.resolve(process.cwd(), "locales");

const CANONICAL_LOCALE = "en.json";

interface LocaleFile {
    file: string;
    name: string;
    raw: string;
    parsed: Record<string, unknown> | null;
    parseError: string | null;
}

function loadLocaleFiles(): LocaleFile[] {
    const pattern = path.join(LOCALES_DIR, "*.json").replace(/\\/g, "/");
    const files = globSync(pattern);
    return files.map((file) => {
        const raw = fs.readFileSync(file, "utf8");
        try {
            const parsed = JSON.parse(raw) as Record<string, unknown>;
            return { file, name: path.basename(file), raw, parsed, parseError: null };
        } catch (err) {
            return {
                file,
                name: path.basename(file),
                raw,
                parsed: null,
                parseError: err instanceof Error ? err.message : String(err),
            };
        }
    });
}

function collectKeys(obj: unknown, prefix = "", out: string[] = []): string[] {
    if (obj === null || typeof obj !== "object" || Array.isArray(obj)) {
        out.push(prefix);
        return out;
    }
    for (const [key, value] of Object.entries(obj as Record<string, unknown>)) {
        const next = prefix ? `${prefix}.${key}` : key;
        collectKeys(value, next, out);
    }
    return out;
}

describe("locale JSON validity", () => {
    let locales: LocaleFile[];

    beforeAll(() => {
        locales = loadLocaleFiles();
    });

    it("finds at least one locale file", () => {
        expect(locales.length, `no locale files found in ${LOCALES_DIR}`).toBeGreaterThan(0);
    });

    it("every locale file is valid JSON", () => {
        const malformed = locales.filter((l) => l.parseError !== null);
        const messages = malformed.map((l) => `${l.name}: ${l.parseError}`);
        expect(malformed, `malformed locale files:\n${messages.join("\n")}`).toHaveLength(0);
    });
});

describe("locale key coverage against canonical en.json", () => {
    let locales: LocaleFile[];
    let canonical: LocaleFile | undefined;
    let canonicalKeys: Set<string>;

    beforeAll(() => {
        locales = loadLocaleFiles();
        canonical = locales.find((l) => l.name === CANONICAL_LOCALE);
        canonicalKeys = canonical?.parsed
            ? new Set(collectKeys(canonical.parsed))
            : new Set();
    });

    it("canonical locale en.json exists and is parseable", () => {
        expect(canonical, `${CANONICAL_LOCALE} not found in ${LOCALES_DIR}`).toBeDefined();
        expect(canonical?.parsed, `${CANONICAL_LOCALE} failed to parse`).not.toBeNull();
    });

    it("every non-canonical locale contains every canonical key", () => {
        const others = locales.filter((l) => l.name !== CANONICAL_LOCALE && l.parsed !== null);
        const missing: { locale: string; keys: string[] }[] = [];

        for (const locale of others) {
            const keys = new Set(collectKeys(locale.parsed!));
            const missingForLocale = [...canonicalKeys].filter((k) => !keys.has(k));
            if (missingForLocale.length > 0) {
                missing.push({ locale: locale.name, keys: missingForLocale });
            }
        }

        const report = missing
            .map((m) => `${m.locale} missing ${m.keys.length} key(s):\n  - ${m.keys.join("\n  - ")}`)
            .join("\n\n");
        expect(missing, `locales with missing keys:\n\n${report}`).toHaveLength(0);
    });
});

# Edge Case Test Generation Prompt

A structured prompt template for Claude Code, Cursor, or any coding
agent that can generate Playwright tests from a feature description
plus a list of production errors. Fill the slots in `[BRACKETS]`
before sending.

## Why this prompt exists

A vanilla prompt like "write a Playwright test for the checkout flow"
produces a happy-path test. This prompt constrains the agent to real
failure modes (your Sentry errors) and to explicit edge-case
categories that vanilla prompts skip. The closing assertion-style
constraint blocks the most common failure mode of AI-generated tests:
assertions that only check status codes and never the response body.

## The template

```
You are generating Playwright tests for the [FEATURE_NAME] feature.

HAPPY PATH (already covered by an existing test, do NOT regenerate):
[HAPPY_PATH_DESCRIPTION]

PRODUCTION ERRORS observed in Sentry over the last 30 days for this
feature. Each line is a real exception users hit. Generate one
Playwright test that reproduces each one:

[SENTRY_ERROR_LIST]
  - Example: "TypeError: Cannot read properties of undefined (reading 'id') at /checkout/confirm"
  - Example: "ValidationError: quantity must be <= 99 at /api/cart/update"
  - Example: "URIError: URI malformed at /api/uploads/download"

ADDITIONAL EDGE-CASE CATEGORIES to cover beyond the Sentry list. For
each category, generate at least one Playwright test:

  1. Empty input. Test every required text field with an empty
     string. Assert a visible error message AND assert the form
     submit endpoint is never POSTed.

  2. Maximum-length input. For every text field with a documented
     max length N, test exactly N characters (must pass) and N+1
     characters (must fail with a visible error).

  3. Boundary value. For every numeric field with a valid range
     [min, max], test min, max, min-1, and max+1. min and max must
     pass; min-1 and max+1 must fail with a visible error.

  4. Special characters in filenames and text. Test inputs
     containing parentheses, spaces, accented characters, and
     trailing whitespace. Assert the value round-trips correctly
     through any persistence and display layer.

  5. Malformed input. Test inputs that look valid but break parsers:
     a quantity of "1e5", a date of "2024-13-45", a JSON payload
     with a trailing comma. Assert the server returns a 4xx with a
     useful error message, NOT a 5xx.

  6. Expired session. Test the flow with a session token that has
     been expired server-side. Assert the user is redirected to
     login and the in-flight action is NOT silently dropped.

ASSERTION-STYLE REQUIREMENT (this is non-negotiable):

  Every assertion must check the CONTENT of the response, not just
  the status code. A test that asserts `response.status() === 200`
  is not an edge case test; it is a smoke test. For each test you
  generate:

    - If the test expects success, assert the response body, the
      rendered DOM, or the database/state side-effect contains the
      specific value you expect.
    - If the test expects failure, assert the visible error message
      text and assert that the side-effect (e.g. database write,
      outbound API call) did NOT occur.
    - For asynchronous bugs (race conditions, eventual consistency),
      use Playwright's auto-retrying assertions (`expect(...).toHaveText`)
      or `waitForResponse`. Never use a fixed `setTimeout`.

OUTPUT FORMAT:
  - One `test.describe` block per category.
  - One `test(...)` per case.
  - All tests in TypeScript using `@playwright/test`.
  - File header comment listing which Sentry error or category each
    test addresses, so the reviewer can audit coverage.
```

## How to use it

1. Run the [Sentry issue-grouping SQL](../queries/sentry-issue-grouping.sql)
   against your events warehouse. Paste the top 10 rows into
   `[SENTRY_ERROR_LIST]`.
2. Fill `[FEATURE_NAME]` and `[HAPPY_PATH_DESCRIPTION]` with the
   feature you want covered.
3. Hand the resulting prompt to your coding agent.
4. Review the generated tests against the three known gaps:
   - Boundary tests whose assertion is just "value accepted".
   - Assertions that never fail (often on async or race code paths).
   - Cases the agent skipped because they require out-of-band setup
     (DB seed, mocked third-party, feature flag). Add those by hand.