Test Data Provisioning: From Fixtures to Throwaway Tenants

Two engineers open PRs against the same feature an hour apart. Both PRs touch the checkout flow. Both preview environments point at the same shared staging database, because nobody ever decided otherwise, it's just the database that existed when the app was small. Engineer A seeds a test order to check a refund path. Engineer B's PR, previewing five minutes later, shows that same order in a customer list it was never supposed to appear in. Nobody broke anything. The data just wasn't provisioned to be theirs alone.

That's the failure mode this post is actually about. Not "do we have realistic test data" (most teams do, eventually) but "does this preview own the data it's looking at, or is it borrowing someone else's." Every team answers that question by climbing a ladder, whether anyone planned the climb or not. This post is that ladder: five rungs, what each one buys you, where it breaks, and what pushes you to the next one.

One thing this post is not: a guide to generating or masking test data. If you need the enterprise landscape, synthetic generation, PII masking, subsetting, the test data management guide covers that ground, and our post on test data generation approaches covers the "how do you make the data itself" question in depth. This post assumes you already have data worth using and asks a narrower question: how does it get into the environment where a test or a preview actually runs.

Rung One: Hardcoded Fixtures

The first rung is the one nobody chooses on purpose. A test needs a user, so someone writes a literal object inline: an email, a name, a plan tier, right there in the test file. It works. It's also the fastest rung to write and the first one to fall apart.

Hardcoded fixtures are fine for pure logic tests where the shape of the data barely matters. The trouble starts the moment your schema changes. Add a required column, rename a field, introduce a new enum value, and every hardcoded literal that doesn't account for it either fails loudly or, worse, passes while testing against a shape your production database no longer produces. Nobody's job is to keep those literals current, so nobody does, until a test suite full of green checkmarks is quietly testing yesterday's schema.

The realism problem compounds it. A hardcoded user object rarely reflects the edge cases production actually has: the customer with three overlapping subscriptions, the order placed in a currency your checkout form doesn't default to. Fixtures test the shape you imagined, not the shape you shipped.

Rung Two: Factories and Builders

The second rung fixes the maintenance problem, not the isolation problem. Factory functions, the FactoryBot or Faker-style pattern, build objects programmatically instead of listing every field by hand. Call a builder for a user, override the two fields your test actually cares about, and let the factory fill in sane defaults for everything else.

This is a real improvement. When the schema adds a field, you update the factory once, not every test file that constructs a user. Factories also compose: a factory for an order can call a factory for a customer, which calls a factory for an account, and the whole tree stays consistent without anyone hand-maintaining the relationships.

What factories don't solve is where the data lives. A factory typically builds an in-memory object or inserts into whatever database connection the test happens to be using, which for most teams is still one shared instance. Two test runs calling the same factory against the same database can still collide, just with less boilerplate than rung one. Factories make data easier to construct. They don't make it belong to anyone.

Rung Three: Seeded Datasets

The third rung is where teams usually get serious about realism. Instead of building objects one test at a time, a seed script loads a known baseline into a real database: a set of customers, orders, and edge cases that resembles what a demo, a QA pass, or a preview actually needs to look believable.

Seeded datasets are a genuine step up. They're shared, versioned, and reviewable, which means the whole team is testing against the same known-good starting point instead of whatever a factory happened to generate that day. A seed script can encode the tricky cases on purpose: the customer with a failed payment mid-cycle, the account that's over its usage limit. That's realism a factory's random defaults won't reliably produce.

The failure mode is the word "shared." One seed script, run once, backing every preview and every CI job, means every one of those runs is reading and writing the same rows. The scenario that opened this post, two PRs colliding on one order, is a seeded-dataset failure as often as it's anything else. The seed gave both previews realistic data. It didn't give either of them their own copy of it.

A seed can make data realistic without making it private to a run. Isolation starts when each preview receives its own data boundary.

Rung Four: Database Branches

The fourth rung finally addresses the sharing problem directly, by giving each preview its own copy instead of a shared one. A database branch, in the Neon or Supabase sense, is a copy-on-write fork: instant, cheap until you write to it, and a full snapshot of whatever was in the parent database at fork time.

This is where isolation actually starts. A branch-per-preview setup means Engineer A and Engineer B from the opening scenario would each get their own fork, and neither one's seeded order would show up in the other's environment. That's a real fix, and it's fast enough to wire into CI without anyone noticing the extra step.

The catch is what a branch inherits. A branch is a copy of the parent database exactly as it was, tenants and all, boundaries and all. If your production data model already separates tenants cleanly, a branch of it is clean too. If it doesn't, forking it just gives you two copies of the same gap. We've written separately about where branching and tenant isolation diverge, because the two get conflated constantly and they answer different questions: a branch answers "give me a copy," not "who's allowed to see what's in it."

Rung Five: Throwaway Tenants

The fifth rung moves the isolation boundary up a layer, from the database to the application. Instead of forking a shared schema and hoping the tenant boundary inside it holds, a throwaway tenant spins up as its own isolated unit at the application layer: seeded, tested against, then discarded when the run ends. Nothing about it is shared with any other test run, because nothing about it was meant to outlive one.

This is the rung that actually matches how most SaaS applications think about their own data in production. If your app already reasons about the world in terms of tenants, testing across that same tenant boundary is the natural place to also draw the line between one preview's data and the next one's. A throwaway tenant isn't a copy of production with a boundary baked in already. It's a fresh boundary, provisioned on demand, that happens to look and behave exactly like a real one while it exists.

The tradeoff is obvious once you say it out loud: spinning up, seeding, and tearing down a tenant per test run is real infrastructure. Someone has to build the provisioning path that creates a throwaway tenant on demand, decide what "seeded" means for that tenant, run the test, and then actually tear it down instead of letting it linger as an orphaned row in a table nobody audits. That's not a schema decision anymore. It's a service you're now operating.

A branch copies the parent's isolation boundary, good or bad. A throwaway tenant draws a fresh boundary at the application layer, then discards it when the run ends.

How Autonoma Provisions Isolated Data for Every Preview

The pain climbing this ladder points at is consistent: the higher the rung, the more provisioning machinery a team has to build before a single preview is trustworthy. Wiring branch creation into CI, deciding what "seeded" means for a fresh fork, or standing up a service that can spin up, seed, and tear down a tenant on demand, none of that is the product. All of it has to work correctly before anyone can trust what a preview shows them.

PreviewKit, Autonoma's managed preview-environment product, exists for exactly that gap. It provisions a full-stack, isolated preview per pull request, backend, database, and services included, so a new PR gets a clean, disposable environment without your team building the seeding and teardown pipeline first. In ladder terms, PreviewKit operates the top rungs, the database-branch or throwaway-tenant machinery, so your team's job shrinks to deciding what "seeded correctly" means for your domain, not building the plumbing that gets data there.

Provisioning clean data is only half the problem, though. Once a preview exists with isolated, realistic data sitting in it, something still has to verify the application behaves correctly against that data, not just that the environment came up clean. That's where the platform's testing agents pick up: a Planner agent reads your codebase and plans test cases, including the database state each scenario needs, against the live preview. An Executor agent runs those tests. A Reviewer agent separates a real bug from an agent error or a plan mismatch. A Diffs Agent keeps the suite current as your schema changes, so coverage doesn't quietly rot the way a hand-maintained seed script does once the person who wrote it moves on.

Each rung buys more isolation and realism, and each one hands the team more provisioning machinery to build and keep running.

Choosing How High to Climb

None of the five rungs is universally correct, and climbing higher isn't automatically the right move for every team. A five-person startup testing an internal admin tool has no business operating a throwaway-tenant provisioning service; seeded datasets are plenty. A twenty-engineer SaaS company shipping ten PRs a day to a multi-tenant product that customers pay real money for is a different story, and staying on rung three there isn't caution, it's the reason two PRs keep colliding on the same seeded order.

The honest version of "which rung should we be on" is really two questions. First, what does a realistic test actually need to catch the bugs that matter to your product, which is a judgment call about your domain, not a provisioning decision at all. Second, once you know that, do you want to be the team that builds and operates the branch-creation pipeline or the tenant-provisioning service that gets you there, or the team that points a managed platform at the problem and spends that engineering time on the product instead. Both are legitimate answers. The mistake is not choosing on purpose, and ending up three rungs up a ladder nobody decided to climb.

Go back to the two engineers from the opening scenario. The fix for their collision was never "write better fixtures" or "seed more carefully." It was recognizing that a shared database was doing a job it was never built to do, naming which rung would actually solve it, and then deciding who was going to build and operate the machinery that rung requires. That decision is worth making explicitly, in writing, before the next collision makes it for you.

For teams that need that isolation without taking on the provisioning system themselves, PreviewKit, Autonoma's managed preview-environment product, is the relevant operational layer. It provisions a full-stack preview for each pull request, including the backend, database, and services, so the team can validate the application against an isolated runtime instead of a shared staging database. Your team still defines the data and scenarios that matter to the product; PreviewKit supplies the per-PR environment in which that work can run without another preview changing its state.

FAQ