Why Multi-Tenant SaaS Testing Starts with Data Isolation

Every multi-tenant SaaS incident report reads the same way. A support ticket lands: "I can see an invoice that isn't mine." The team scrambles, patches the one route that leaked, ships a hotfix, and adds exactly one test for the one endpoint that broke.

Six weeks later, a different endpoint leaks a different resource, because the fix was a patch and the testing strategy was still "we'll get to isolation once the feature roadmap slows down." It never slows down. The pattern repeats because isolation is treated as a property you verify once you feel finished building, instead of a property you test continuously, the same way you test any other feature.

That's backwards. In a multi-tenant SaaS application, the tenant boundary is not a feature. It's the thing the entire business model depends on. A bug in checkout costs you a transaction. A bug in tenant isolation costs you a customer, a disclosure notice, and possibly a SOC 2 finding. The rest of this guide treats data isolation as the starting point for multi-tenant testing rather than an afterthought bolted on once the leaked-invoice ticket forces the issue.

The Four Things Multi-Tenant Testing Actually Has to Cover

Frameworks like AWS's SaaS Lens are useful for naming the concepts a multi-tenant architecture has to get right. Where they stop short is code: they describe the shape of the problem and leave the verification to you. Before getting to the runnable part, it's worth being precise about what "multi-tenant testing" actually spans, because teams that only test one of these four end up confident about the wrong thing.

Tenant isolation and noisy neighbor: the boundary itself, and the boundary under load

Tenant isolation is the guarantee that Tenant A's requests, however they're crafted, cannot read, write, or enumerate Tenant B's data. It fails in three ways: a missing WHERE tenant_id = ? clause, an authorization check that confirms the requester is logged in but never confirms the resource is theirs, and an object ID guessable enough that Tenant A can just try neighboring IDs. The first two are what the leakage suite below is built to catch.

Isolation testing usually stops at data, but a boundary that leaks performance is still broken, just quieter. Noisy-neighbor testing asks whether Tenant A's usage (a bulk import, a runaway report, a webhook storm) can degrade latency for Tenant B, who did nothing wrong; the usual causes are unindexed tenant-scoped queries, shared connection pools with no per-tenant caps, and workers processing jobs in submission order instead of fair-share order across tenants.

Tier and quota enforcement, and the boundary changing over time

Most multi-tenant SaaS products aren't flat: tenants sit on tiers, tiers carry quotas (seats, API calls per minute, storage, export rows), and enforcement has to happen per tenant, not globally. A rate limiter keyed by IP instead of tenant ID lets one abusive tenant exhaust a bucket every other tenant on that gateway shares. A seat limit enforced only client-side is trivially bypassed by a direct API call.

Isolation also isn't static: every new tenant is a boundary being created, and every canceled tenant is one that has to be torn down completely. A common onboarding bug is a fresh tenant inheriting a cloned template workspace, template rows included. A common offboarding bug is stale foreign keys or cached permissions leaving a ghost boundary a resurrected account can walk back through.

Those four concepts are the map. The next section is the part the existing SERP for this topic doesn't have: the code.

The Cross-Tenant Leakage Test Suite: UI, API, and Background Jobs

The core pattern behind every cross-tenant leakage test is the same, and it's worth stating plainly because it's the thing to automate everywhere, not just in one corner of the app: authenticate as Tenant A, then request a resource that belongs to Tenant B by its ID, and assert the response is a 403 or a 404, never a 200. What makes this hard in practice isn't the assertion. It's remembering to run that assertion across every surface where a tenant-scoped ID can be requested, which is more surfaces than most teams check.

Three surfaces cover the vast majority of real leakage incidents. The UI surface catches deep links and client-side routing that never re-validates ownership. The API surface catches the actual authorization gap, since the UI is often just a well-behaved client sitting in front of a poorly-defended API. The background job surface catches the one nobody thinks to test: an async worker that receives a tenant ID as untrusted input and never re-checks it against the record it's about to touch.

The same assertion runs on every surface a tenant-scoped ID can be requested: authenticate as Tenant A, ask for Tenant B by ID, and require a 403 or 404 rather than a 200.

Fixtures: seeding two tenants you can actually test against

Every leakage test needs two real tenants with real, non-overlapping resources, seeded and torn down the same way every run. Here's the seed script that creates Tenant A ("Acme") and Tenant B ("Globex"), each with a user and an invoice, and prints the IDs the rest of the suite consumes:

import {
  adminFetch,
  BASE_URL,
  DEFAULT_TENANT_PASSWORD,
  requireAdminToken,
  TENANT_DEFINITIONS,
  writeSeedState,
  type SeedState,
  type TenantKey,
  type TenantSeed,
} from "./shared";

/**
 * Seed two fully isolated tenants (Acme + Globex), each with one user and one
 * invoice, via the admin API. Created IDs are written to the seed-state file
 * AND printed as JSON to stdout so the UI/API leakage tests can consume them.
 *
 * Uses built-in Node `fetch` only. Run with: `npx tsx fixtures/seedTenants.ts`.
 */

interface CreatedResource {
  id: string;
}

async function postJson(path: string, body: unknown, label: string): Promise<CreatedResource> {
  const res = await adminFetch(path, { method: "POST", body: JSON.stringify(body) });
  if (!res.ok) {
    const detail = await res.text().catch(() => "");
    throw new Error(`Failed to create ${label} (POST ${path}): ${res.status} ${res.statusText} ${detail}`);
  }
  const created = (await res.json()) as Partial<CreatedResource>;
  if (!created?.id) {
    throw new Error(`Admin API returned no id when creating ${label} (POST ${path}).`);
  }
  return { id: created.id };
}

async function seedTenant(def: (typeof TENANT_DEFINITIONS)[number]): Promise<TenantSeed> {
  const tenant = await postJson(
    "/admin/tenants",
    { name: def.name, slug: def.slug },
    `tenant "${def.slug}"`,
  );

  const user = await postJson(
    `/admin/tenants/${tenant.id}/users`,
    { email: def.userEmail, password: DEFAULT_TENANT_PASSWORD },
    `user for tenant "${def.slug}"`,
  );

  const invoice = await postJson(
    `/admin/tenants/${tenant.id}/invoices`,
    { number: def.invoiceNumber, amountCents: def.amountCents, currency: def.currency },
    `invoice for tenant "${def.slug}"`,
  );

  return {
    key: def.key,
    id: tenant.id,
    name: def.name,
    slug: def.slug,
    user: { id: user.id, email: def.userEmail, password: DEFAULT_TENANT_PASSWORD },
    invoice: {
      id: invoice.id,
      number: def.invoiceNumber,
      amountCents: def.amountCents,
      currency: def.currency,
    },
  };
}

async function main(): Promise<void> {
  requireAdminToken();

  const tenants = {} as Record<TenantKey, TenantSeed>;
  for (const def of TENANT_DEFINITIONS) {
    tenants[def.key] = await seedTenant(def);
  }

  const state: SeedState = {
    createdAt: new Date().toISOString(),
    baseUrl: BASE_URL,
    tenants,
  };

  writeSeedState(state);

  // Emit the created IDs as JSON for the suite (and CI logs) to consume.
  process.stdout.write(`${JSON.stringify(state, null, 2)}\n`);
}

main().catch((error: unknown) => {
  const message = error instanceof Error ? error.message : String(error);
  process.stderr.write(`[seed] ${message}\n`);
  process.exitCode = 1;
});

Notice the seed script never reuses an ID range between tenants and returns the created IDs explicitly rather than assuming they're sequential. A leakage suite that "guesses" Tenant B's invoice ID by decrementing Tenant A's is testing a lucky guess, not a boundary. Teardown matters just as much as seeding, because a leakage suite that leaves orphaned tenants behind will eventually let a stale Tenant B collide with a freshly seeded one and produce a false pass:

import { adminFetch, clearSeedState, readSeedState, requireAdminToken, type TenantSeed } from "./shared";

/**
 * Delete both seeded tenants and every resource they own (users + invoices)
 * via the admin API, so leakage tests never accumulate stale data.
 *
 * A tenant that is already gone (404) is treated as success, which keeps this
 * safe to run repeatedly and inside a CI `if: always()` step.
 *
 * Uses built-in Node `fetch` only. Run with: `npx tsx fixtures/teardownTenants.ts`.
 */

interface TeardownResult {
  key: string;
  id: string;
  status: number;
  deleted: boolean;
}

async function deleteTenant(tenant: TenantSeed): Promise<TeardownResult> {
  const res = await adminFetch(`/admin/tenants/${tenant.id}`, { method: "DELETE" });

  // 404 means it was already removed; anything else that is not OK is a failure.
  if (!res.ok && res.status !== 404) {
    const detail = await res.text().catch(() => "");
    throw new Error(
      `Failed to delete tenant "${tenant.slug}" (DELETE /admin/tenants/${tenant.id}): ` +
        `${res.status} ${res.statusText} ${detail}`,
    );
  }

  return { key: tenant.key, id: tenant.id, status: res.status, deleted: res.status !== 404 };
}

async function main(): Promise<void> {
  requireAdminToken();

  const state = readSeedState();
  const results: TeardownResult[] = [];
  for (const tenant of Object.values(state.tenants)) {
    results.push(await deleteTenant(tenant));
  }

  clearSeedState();
  process.stdout.write(`${JSON.stringify({ tornDown: results }, null, 2)}\n`);
}

main().catch((error: unknown) => {
  const message = error instanceof Error ? error.message : String(error);
  process.stderr.write(`[teardown] ${message}\n`);
  process.exitCode = 1;
});

The UI surface: deep links into another tenant's data

The UI leakage test logs in as Tenant A and then navigates directly to the URL for Tenant B's invoice, the way a curious (or malicious) user would if they ever saw that URL in a screenshot, a shared bookmark, or a browser history sync. The assertion isn't "the page doesn't render Tenant B's numbers." It's "the app shows a not-found or access-denied state," because a UI that silently redirects to an empty dashboard without an explicit denial is easy to miss in a manual review and easy to regress without anyone noticing:

import { expect, test } from "@playwright/test";
import { BASE_URL, readSeedState, type SeedState } from "../../fixtures/shared";

/**
 * UI cross-tenant leakage.
 *
 * Log in through the UI as Tenant A, then navigate directly to Tenant B's
 * invoice URL. The app must render a not-found / access-denied state instead of
 * Tenant B's invoice data. This catches leaks that only appear in the rendered
 * page (e.g. a client-side route that fetches by ID without a tenant guard).
 *
 * Requires a running app at BASE_URL and a prior `npm run seed`.
 */

let state: SeedState;

test.beforeAll(() => {
  state = readSeedState();
});

test.beforeEach(async ({ page }) => {
  // Authenticate as Tenant A's user.
  await page.goto(`${BASE_URL}/login`);
  await page.locator('input[name="email"]').fill(state.tenants.acme.user.email);
  await page.locator('input[name="password"]').fill(state.tenants.acme.user.password);
  await page.locator('button[type="submit"]').click();
  await page.waitForLoadState("networkidle");
});

test.describe("UI cross-tenant isolation", () => {
  test("Tenant A cannot open Tenant B's invoice by direct URL", async ({ page }) => {
    const globexInvoice = state.tenants.globex.invoice;

    const response = await page.goto(`${BASE_URL}/invoices/${globexInvoice.id}`);
    await page.waitForLoadState("networkidle");

    // Tenant B's invoice number must never render on Tenant A's screen.
    await expect(page.getByText(globexInvoice.number)).toHaveCount(0);

    // The page must present an explicit denial. Accept either a non-2xx
    // server response OR a client-rendered access-denied / not-found state.
    const statusDenied = response !== null && response.status() >= 400;
    const deniedText = page.getByText(/not found|access denied|forbidden|no access|not authorized/i);
    const textDenied = await deniedText
      .first()
      .isVisible()
      .catch(() => false);

    expect(statusDenied || textDenied).toBeTruthy();
  });

  test("Tenant A CAN open its own invoice (control)", async ({ page }) => {
    const acmeInvoice = state.tenants.acme.invoice;

    await page.goto(`${BASE_URL}/invoices/${acmeInvoice.id}`);
    await page.waitForLoadState("networkidle");

    await expect(page.getByText(acmeInvoice.number).first()).toBeVisible();
  });
});

The API surface: direct requests to another tenant's resource IDs

This is the test that actually proves the authorization boundary, because it bypasses the UI entirely and goes straight at the endpoints. Authenticate as Tenant A, then fire requests at Tenant B's invoice, user, and export endpoints using Tenant B's real IDs from the seed step. Every one of them should come back 403 or 404. A 200 here means the UI is the only thing standing between Tenant A and Tenant B's data, which is not a boundary, it's an inconvenience:

import { beforeAll, describe, expect, it } from "vitest";
import { BASE_URL, readSeedState, type SeedState } from "../../fixtures/shared";

/**
 * API cross-tenant leakage.
 *
 * Authenticate as Tenant A's user, then directly request Tenant B's invoice,
 * user, and export endpoints by ID. Each MUST return 403 or 404 and MUST NEVER
 * return 200 with another tenant's data. Direct object references (IDOR) are
 * the classic multi-tenant leak, so we probe them explicitly.
 *
 * Requires a running app at BASE_URL and a prior `npm run seed`.
 */

/** Cross-tenant access is only acceptable as an explicit denial. */
const DENIED_STATUSES = [401, 403, 404] as const;

async function login(email: string, password: string): Promise<string> {
  const res = await fetch(`${BASE_URL}/api/auth/login`, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ email, password }),
  });
  if (!res.ok) {
    const detail = await res.text().catch(() => "");
    throw new Error(`Login failed for ${email}: ${res.status} ${res.statusText} ${detail}`);
  }
  const body = (await res.json()) as { token?: string };
  if (!body.token) {
    throw new Error(`Login for ${email} returned no token.`);
  }
  return body.token;
}

function authedGet(path: string, token: string): Promise<Response> {
  return fetch(`${BASE_URL}${path}`, {
    headers: { authorization: `Bearer ${token}`, accept: "application/json" },
  });
}

describe("API cross-tenant isolation", () => {
  let state: SeedState;
  let tokenA: string;

  beforeAll(async () => {
    state = readSeedState();
    // Log in as Tenant A. Every request below carries Tenant A's credentials.
    tokenA = await login(state.tenants.acme.user.email, state.tenants.acme.user.password);
  });

  it("lets Tenant A read its OWN invoice (control)", async () => {
    const res = await authedGet(`/api/invoices/${state.tenants.acme.invoice.id}`, tokenA);
    expect(res.status).toBe(200);
  });

  it("blocks Tenant A from reading Tenant B's invoice", async () => {
    const targetId = state.tenants.globex.invoice.id;
    const res = await authedGet(`/api/invoices/${targetId}`, tokenA);

    expect(res.status).not.toBe(200);
    expect(DENIED_STATUSES).toContain(res.status);

    // Defense in depth: even if a proxy rewrote the status, the body must not
    // carry Globex's invoice number.
    const text = await res.text().catch(() => "");
    expect(text).not.toContain(state.tenants.globex.invoice.number);
  });

  it("blocks Tenant A from reading Tenant B's user", async () => {
    const res = await authedGet(`/api/users/${state.tenants.globex.user.id}`, tokenA);

    expect(res.status).not.toBe(200);
    expect(DENIED_STATUSES).toContain(res.status);

    const text = await res.text().catch(() => "");
    expect(text).not.toContain(state.tenants.globex.user.email);
  });

  it("blocks Tenant A from exporting Tenant B's invoice", async () => {
    const res = await authedGet(`/api/exports/${state.tenants.globex.invoice.id}`, tokenA);

    expect(res.status).not.toBe(200);
    expect(DENIED_STATUSES).toContain(res.status);
  });
});

Background jobs: the surface everyone forgets

This is the gap that separates a real multi-tenant leakage suite from a superficial one. Background workers (export generators, webhook dispatchers, scheduled reports, billing reconciliation jobs) usually receive a tenant ID as part of their job payload, and that ID is exactly as trustworthy as whatever enqueued it. If a job is ever enqueued with an incorrect or tampered tenant ID (a bug in the enqueueing code, a replayed webhook, a race during a tenant merge), does the job's own query layer independently re-verify tenant scoping, or does it trust the payload? The test enqueues an export job for Tenant A but deliberately points one of its inputs at a Tenant B resource ID, and asserts the job either fails closed or returns zero rows for the out-of-scope resource, never Tenant B's data:

import { beforeEach, describe, expect, it } from "vitest";
import {
  InvoiceStore,
  runExportJob,
  TenantIsolationError,
  TestJobQueue,
  type ExportJob,
} from "../../src/jobs/exportJob";

/**
 * Background-job cross-tenant leakage.
 *
 * We enqueue an export job scoped to Tenant A but deliberately point one of its
 * inputs at a Tenant B resource ID, then assert the job either fails closed
 * (strict mode) or returns zero rows for the out-of-scope resource (lenient
 * mode). Either way, Tenant B's data must never appear in Tenant A's export.
 *
 * This suite is fully self-contained: it needs no running server, so it also
 * runs standalone via `npm run test:jobs`.
 */

const TENANT_A = "tenant_acme";
const TENANT_B = "tenant_globex";

const ACME_INVOICE_ID = "inv_acme_1001";
const GLOBEX_INVOICE_ID = "inv_globex_2001";

function buildStore(): InvoiceStore {
  return new InvoiceStore()
    .add({ id: ACME_INVOICE_ID, tenantId: TENANT_A, number: "ACME-1001", amountCents: 129_900 })
    .add({ id: GLOBEX_INVOICE_ID, tenantId: TENANT_B, number: "GLOBEX-2001", amountCents: 543_200 });
}

describe("background-job cross-tenant isolation", () => {
  let store: InvoiceStore;
  let queue: TestJobQueue;

  beforeEach(() => {
    store = buildStore();
    queue = new TestJobQueue();
  });

  it("fails closed when a Tenant A job reaches for a Tenant B invoice (strict)", () => {
    const job: ExportJob = {
      id: "job_export_a_1",
      tenantId: TENANT_A,
      // One legitimate input plus one deliberately out-of-scope Tenant B ID.
      invoiceIds: [ACME_INVOICE_ID, GLOBEX_INVOICE_ID],
    };
    queue.enqueue(job);

    const results = queue.drain(store, { strict: true });
    expect(results).toHaveLength(1);
    const result = results[0]!;

    expect(result.status).toBe("failed");
    expect(result.rows).toHaveLength(0);
    expect(result.skipped).toEqual([GLOBEX_INVOICE_ID]);
    expect(result.error).toContain(GLOBEX_INVOICE_ID);
  });

  it("drops out-of-scope rows and never emits Tenant B data (lenient)", () => {
    const job: ExportJob = {
      id: "job_export_a_2",
      tenantId: TENANT_A,
      invoiceIds: [ACME_INVOICE_ID, GLOBEX_INVOICE_ID],
    };

    const result = runExportJob(job, store, { strict: false });

    expect(result.status).toBe("completed");
    expect(result.rows).toHaveLength(1);
    expect(result.rows[0]?.invoiceId).toBe(ACME_INVOICE_ID);
    expect(result.rows.every((row) => row.tenantId === TENANT_A)).toBe(true);
    // The Tenant B invoice is out of scope: zero rows, recorded as skipped.
    expect(result.rows.some((row) => row.invoiceId === GLOBEX_INVOICE_ID)).toBe(false);
    expect(result.skipped).toContain(GLOBEX_INVOICE_ID);
  });

  it("returns zero rows for a job whose only input is out of scope", () => {
    const job: ExportJob = {
      id: "job_export_a_3",
      tenantId: TENANT_A,
      invoiceIds: [GLOBEX_INVOICE_ID],
    };

    const result = runExportJob(job, store, { strict: false });

    expect(result.rows).toHaveLength(0);
    expect(result.skipped).toEqual([GLOBEX_INVOICE_ID]);
  });

  it("still exports the tenant's own invoices unharmed", () => {
    const job: ExportJob = {
      id: "job_export_a_4",
      tenantId: TENANT_A,
      invoiceIds: [ACME_INVOICE_ID],
    };

    const result = runExportJob(job, store);

    expect(result.status).toBe("completed");
    expect(result.rows).toHaveLength(1);
    expect(result.rows[0]?.number).toBe("ACME-1001");
    expect(result.skipped).toHaveLength(0);
  });

  it("TenantIsolationError names the offending job and resources", () => {
    const error = new TenantIsolationError("job_export_a_1", [GLOBEX_INVOICE_ID]);
    expect(error.name).toBe("TenantIsolationError");
    expect(error.jobId).toBe("job_export_a_1");
    expect(error.offendingInvoiceIds).toEqual([GLOBEX_INVOICE_ID]);
  });
});

Wiring it into CI

None of this matters if it only runs on someone's laptop the week it was written. The leakage suite needs to seed its own tenants, run all three surfaces, and tear down, on every pull request, with the teardown step guaranteed to run even if an assertion fails partway through:

name: cross-tenant-leakage

# Seed two isolated tenants, run the UI + API + background-job leakage suites on
# every pull request, then tear the tenants down no matter what happened.
on:
  pull_request:
  workflow_dispatch:

concurrency:
  group: cross-tenant-leakage-${{ github.ref }}
  cancel-in-progress: true

jobs:
  isolation:
    runs-on: ubuntu-latest
    env:
      BASE_URL: ${{ secrets.BASE_URL }}
      ADMIN_API_TOKEN: ${{ secrets.ADMIN_API_TOKEN }}
      TENANT_USER_PASSWORD: ${{ secrets.TENANT_USER_PASSWORD }}
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm install

      - name: Install Playwright browser
        run: npx playwright install --with-deps chromium

      - name: Seed two isolated tenants (Acme + Globex)
        run: npm run seed

      - name: API cross-tenant leakage
        run: npm run test:api

      - name: Background-job cross-tenant leakage
        run: npm run test:jobs

      - name: UI cross-tenant leakage
        run: npm run test:e2e

      - name: Tear down tenants
        if: always()
        run: npm run teardown

Wire this into the same pipeline that runs your feature tests. A cross-tenant leakage suite that only runs manually before a release is a leakage suite that catches the bug two weeks after it shipped instead of before it merged.

Provisioning Test Data for Multi-Tenant Testing: Four Strategies Compared

A leakage suite is only as good as the tenant data it runs against. Seed two tenants with identical, trivially distinguishable records ("Tenant A user," "Tenant B user") and the suite will pass even when the underlying query has a subtler scoping bug that only surfaces against realistic, overlapping data shapes. This is the part every multi-tenant testing guide skips entirely, and it's the actual moat: how do you get realistic, isolated tenant data into a test environment without hand-rolling a seeding script for every new resource type?

Four strategies show up in practice, and none of them wins on every axis. The strongest strategy for most teams is a per-preview, tenant-scoped dataset seeded through code, because it's the only approach that combines real isolation strength with fast, reliable teardown.

Synthetic generation is the safest default: fast, no prod data touched, but only as realistic as whoever wrote the factory definitions. Subset-and-mask gets real data shapes at the cost of a masking pipeline that needs its own audit. Per-tenant seed scripts are what most teams actually run today, and they work until the tenth engineer adds an eleventh script nobody can tell is canonical.

Scored on the two axes that decide provisioning in practice, the per-preview tenant-scoped dataset sits where strong isolation meets fast, reliable teardown.

The fourth option, a branch or tenant-scoped dataset per run, is where provisioning overlaps the preview-environment decision every Vercel or Netlify team eventually hits: fork a database branch per PR, or scope a shared database with a disposable tenant ID per run. Which one actually holds up as an isolation boundary rather than just a data-freshness trick is its own decision, worked through directly, side by side, in database branching vs tenant isolation. For the seeding mechanics specifically, look at how to isolate test data per tenant and multi-tenant test data isolation. The existing guide to database branching covers the copy-on-write mechanics this table only scores.

Whichever strategy you pick, the leakage suite doesn't change. What changes is how expensive it is to keep two clean, isolated tenants seeded and torn down for every single PR, and that cost is where most teams quietly give up and start sharing a database across previews.

How Autonoma Provisions Isolated Tenant Data for Every PR

Here's the pattern that shows up in nearly every team that reaches this point: the leakage suite above gets written once, by whoever got burned by the incident, and it works. Then six months pass. The seed script drifts from the schema (a new required field breaks seedTenants.ts silently), the teardown step starts leaving orphaned tenants after a flaky CI run, and nobody notices until the leakage suite's own test data starts colliding with itself. Hand-building cross-tenant isolation tests is the easy part. Hand-maintaining two clean, isolated tenants and their teardown, for every PR, indefinitely, across a codebase that keeps adding tenant-scoped tables, is the part that quietly turns into a second job nobody signed up for.

This is exactly the gap Autonoma's PreviewKit closes. PreviewKit provisions a full, isolated preview environment (backend, database, and services) per pull request, so "seed two tenants and tear them down cleanly" isn't a script someone owns forever, it's a property of the environment itself: each PR gets its own database, which means Tenant A and Tenant B in your leakage suite are never sharing storage with the PR that merged last week, and a flaky teardown on one PR can never contaminate the next one. That isolation-by-construction is what a hand-rolled seed-and-teardown script can only approximate, because a script is one missed edge case away from leaving orphaned rows behind.

On top of that isolated preview, Autonoma's Planner agent reads the codebase, including the tenant model and the routes that scope by tenant ID, and plans test cases against it, generating the endpoints needed to put that PR's database into the right multi-tenant state (two tenants, non-overlapping resources, the same shape as the fixtures earlier in this article) automatically rather than requiring a hand-maintained seed script. When a PR adds a new tenant-scoped table, the Planner reads that table the same way it reads every other route and model, so the test data it generates for that PR already reflects the schema as it exists right now, not as it existed when someone last updated a fixture file.

Each pull request gets isolated tenant data, then the Planner, Executor, Reviewer, and Diffs Agent keep cross-tenant coverage aligned with the current schema.

From there, the Executor agent runs the planned cross-tenant assertions against the live preview, the same UI-deep-link and direct-API-request pattern described above, but driving the actual application rather than a script someone has to keep in sync.

The Reviewer agent then classifies what comes back: a real cross-tenant leak, an agent error worth ignoring, or a test-plan mismatch against a legitimate schema change, which is the distinction that keeps a growing test suite from drowning a team in false alarms every time the tenant model evolves.

The Diffs Agent is what actually solves the maintenance problem this section opened with: on every PR, it re-reads the diff, adds new leakage assertions for new tenant-scoped tables, deprecates assertions for tables that were removed, and updates the plan for tables that changed shape, so a renamed tenant_id column doesn't quietly leave a hole in the suite the way a stale hand-written seed script would. Each of the four agents runs through its own verification layers, so a passing result means the assertion actually held against the live preview, not that an agent happened to click through a plausible-looking path.

Mapped against the three surfaces this article covers: the Planner's generated DB state replaces the fixtures section, the Executor replaces the UI and API test runs, the Reviewer replaces the manual triage of a failing leakage test, and the Diffs Agent replaces the CI-wiring discipline of keeping the suite current as the schema moves. Background job leakage is the one surface that still benefits from an explicit assertion in your own job-processing code, since a job's tenant scoping is enforced inside application logic an external agent can verify but shouldn't be silently rewriting. If you're comparing this against building the equivalent yourself with a managed ephemeral-environment vendor, ephemeral environment platforms compared shows where the general-purpose ones stop short specifically on the database layer, which is the exact gap PreviewKit is built to close.

None of this replaces the judgment behind the leakage tests in this article. It replaces the part of the job that has nothing to do with testing and everything to do with keeping isolated, realistic tenant data alive, seeded, and torn down cleanly across every PR, indefinitely, without anyone remembering to.

Worth being precise about what's still yours to own. Autonoma doesn't invent your tenant model or decide what isolation means for your product; the Planner reads the schema and routes you already wrote, which means an application with no tenant scoping at all still has to add that scoping before there's anything for the agents to verify against.

What changes is who maintains the verification once it exists. A hand-written leakage suite is a snapshot of what the codebase looked like the day it was written, and it stays accurate only as long as someone remembers to update it every time the tenant model changes.

A Diffs Agent reading every PR's schema diff doesn't share that failure mode, because it isn't relying on anyone's memory. That's the actual argument for managed preview infrastructure here: not that it tests better than the suite in this article, but that it keeps testing that well on PR two thousand the same way it did on PR one, without a person in the loop remembering to.

If hand-seeding two isolated tenants and hand-maintaining a cross-tenant leakage suite across the UI, API, and background jobs for every single PR sounds like exactly the kind of second job no one signed up for, that is precisely the gap managed preview infrastructure with isolated, per-PR tenant data is built to close.

FAQ