Validation testing checks whether a system does what its users actually need it to do, judging behavior against user intent rather than code against a written spec. It runs at three levels, system, acceptance, and production, each against a different source of intent. That is a different question than verification asks, and it's why validation testing in software testing keeps a human in the loop no matter how much of the checking gets automated.
A checkout flow can pass every automated check in the suite and still be wrong. The response codes are correct. The cart total renders. The confirmation page loads. The order still routes to the wrong fulfillment center, because the routing rule matched a spec written eight months ago and nobody told the test what the business needs today. The code did exactly what it was told. That was never the question that mattered.
This is written for the engineer, SDET, or release owner staring at a build and deciding whether it's fit to ship, or the person who just inherited a validation gate and has to name what it checks. It is not written for someone dividing a QA team's time across a suite-allocation strategy, checking whether an AI-generated suite actually asserts anything real, or picking inputs for a single test case. Those questions sit one level up from this one (allocating a team's testing time across a whole suite), one level to the side (auditing whether an AI-generated suite actually asserts anything real), and one level down (choosing inputs for a single test case).
What Validation Testing Checks (and What It Doesn't)
Validation testing in software testing is the half of testing that checks behavior against intent, not code against spec. Verification asks whether the system was built according to its specification: does the code do what the requirements document says it should do. That question can be answered entirely from inside the repository, because the spec and the code are both artifacts a machine can compare. Validation asks a different question: does the system do what the user actually needs, regardless of what the specification happened to say. A build can pass every verification check against a spec that was wrong from the start, and validation is the only place that gets caught. IEEE 1012 formalizes the same split, describing verification and validation processes as work that determines "whether the development products of a given activity conform to the requirements of that activity and whether the product satisfies its intended use and user needs."
That split, validation testing vs verification testing, is worth stating precisely once before going deep on one half of it. The full verification-vs-validation comparison covers the pairing itself, the classic building-it-right against building-the-right-thing framing, and where each fits across a release. This article stays on one side of that line: what validation looks like once you commit to running it, at every level a product actually gets checked.
The Levels of Validation Testing: System, Acceptance, Production
Validation isn't one check that runs once. It runs at three levels, each asking the same underlying question, is this what the user needed, against a different unit and a different source of intent.
System-Level Validation
System-level validation testing checks the fully assembled application against the intent baked into its own requirements and design, the same does-this-behave-the-way-the-product-was-supposed-to-behave question a QA lead or product owner answers by exercising real user flows top to bottom. The mechanics of exercising those flows belong to system testing; this page treats system-level validation only as the widest of the three intent sources, checked against the product's own design rather than a written criterion or live usage.
Acceptance-Level Validation
Acceptance-level validation testing checks one feature or user story against written acceptance criteria, signed off by the product owner or the customer who wrote those criteria.
Production-Level Validation
Production-level validation checks the shipped system against the intent of real users in the wild, using signals no pre-release environment can produce: usage patterns, support tickets, a beta cohort's actual behavior.
| Level | What's validated | Intent source | Oracle | Signs off |
|---|---|---|---|---|
| System | Full assembled application | Requirements and design docs | Design intent, re-derived | QA lead, product owner |
| Acceptance | One feature or user story | Written acceptance criteria | The criteria itself | Product owner, customer |
| Production | Shipped system, live traffic | Real user behavior | Usage data, feedback | Beta cohort, support, PM |
Each ring answers the same question against a wider unit and a different source of intent.
Acceptance-level validation has its own taxonomy, alpha, beta, contract, regulatory, worth reading on its own; this section only places acceptance inside the wider validation stack. The unscripted version of validation, where a person pokes at a build because something felt off rather than following a script, is exploratory testing; it doesn't belong to any one of the three levels above because it can happen at any of them.
The Test Oracle Problem: Why Validation Stays a Human Call
Every test, at every level above, is actually two separable pieces glued together. Barr, Harman, McMinn, Shahbaz, and Yoo's survey of the field, "The Oracle Problem in Software Testing," is where the harder half gets its name: it describes distinguishing the corresponding desired, correct behaviour from potentially incorrect behavior as the "test oracle problem," and its conclusion is blunt about where that leaves things: "when none of these is completely adequate, the final source of test oracle information remains the human." The procedure drives the system into a state: log in, add an item, submit the form, hit the endpoint. The oracle decides whether the state the procedure produced is the right one. Testing folklore treats a test as a single artifact, but the procedure and the oracle answer completely different questions, and only one of them has gotten cheaper.
The procedure got automated. The oracle is still where a human has to decide.
Automated test generation, whatever is reading the codebase to produce test cases, is very good at the procedure half. It can trace a user flow through the UI, fill in a form, submit a request, and land the system in a specific state, reliably and repeatedly. What it cannot do is originate the answer to whether that state is the right one, because rightness is not a property the code exposes. Nothing in a codebase states what the total on a checkout page is supposed to be for a given cart, in the sense that matters: whether that's the total the business actually wants to charge this customer, today, under this pricing rule. A generated test can drive the checkout flow flawlessly, land on the confirmation page, and assert nothing more meaningful than a confirmation page rendered. It passes. It also proves nothing about whether the number on that page is correct.
Take a concrete version: a subscription upgrade flow with prorated billing. A generated procedure logs in, selects the Pro tier, submits, and lands on the confirmation page. That's a working procedure. Whether the prorated charge for the days remaining in the billing cycle is the amount the business actually intends to charge is invisible to a check that only confirms the page rendered. Somebody encoded that oracle, or nobody tested whether the number was right at all.
Four steps passed. The one question that mattered was never asked.
That's the oracle problem, and it's why validation concentrates human judgment even as procedures get automated. How to write good test assertions is the practical half of this: the assertion is the artifact where the oracle actually lives, encoded as a comparison a machine can check. When that encoding is missing or too weak, a generated test suite can pass every run and still not be testing anything real, which is the same failure mode from the other direction: a fine procedure with no oracle behind it.
How Autonoma runs validation checks and surfaces the judgement calls
The pattern the sections above describe, a cheap-to-generate procedure and a scarce, human-sourced oracle, is exactly the shape of the problem when a team decides to automate as much of validation as it can. Get the split wrong, and automation just re-hides the oracle problem instead of solving it: a tool that claims to validate a flow but only ever checks that the procedure completed is making the same mistake as the blind generated test above, with better tooling wrapped around it.
Our agents at Autonoma read the codebase and drive the running application to handle the procedure half honestly: reaching the checkout page, submitting the upgrade, landing on the state where a question needs to be asked, generated from the routes and components that actually exist, and re-derived automatically by the Diffs Agent when a pull request changes them, so the procedure doesn't go stale the way a hand-written script does. Mapped onto the checkout and subscription examples above, that split holds exactly: the agents can reach the confirmation page or the upgrade receipt every time the build changes, but deciding whether the total charged that day was the total the business intended to charge isn't a call the platform gets to make on its own.
What the platform does not do, and should not claim to do, is originate the oracle for intent. It surfaces the behavior it observed and the diff against a prior run; deciding whether that behavior is what the product was supposed to do next stays with the person who owns the acceptance criteria or the release.
The procedure gets handled, and the call still gets made by a human who knows what the business intends. Beta cohorts, exploratory sessions, and a product owner's sign-off remain the tools for the half of validation that checks intent, and they stay there regardless of how much of the procedure gets generated out from under them.
Where This Leaves the Sign-Off
None of the three levels above collapse into one gate, and none of them get to skip the oracle problem regardless of how much of the procedure is automated. System-level validation still needs someone who understands the product's design intent to look at what a run surfaced and decide if it matches. Acceptance-level validation still needs the person who wrote the criteria to say whether they were met. Production-level validation, the one no pre-release environment can fully substitute for, still needs real usage, a beta cohort, or a support queue to say whether the thing you shipped was the thing people wanted, and no scheduled check against a staging environment replaces that.
That's also where the audience for this page stays narrow. Deciding whether today's build clears its validation gate, or naming the criteria a specific check should run against, is a different job from allocating a QA team's testing budget across a whole suite, or auditing whether an existing AI-generated suite actually asserts anything real once it's already running. Those are legitimate questions. They just live one level up and one level to the side of the one this page answers.
The procedure regenerates on every pull request. The judgment call stays with a person.
What changes, once the procedure half of every level above stops being hand-maintained, is where the sign-off's attention goes. Instead of spending the sprint re-running the same click-through by hand to reach the state where a decision needs to be made, the person who owns the sign-off spends it on the decision itself: does the observed behavior match what the acceptance criteria, the design intent, or the user actually wanted. If you're the one deciding whether today's build clears that gate, or naming the criteria a validation check should run against, that decision is the whole job this article describes, and Autonoma exists to make sure the time goes to the judgment call and not to re-driving the same flow for the tenth time this quarter. The oracle problem doesn't disappear. It just stops competing for time with the procedure that used to eat the whole afternoon.
Frequently Asked Questions
Validation testing checks whether a system does what its users actually need, judging behavior against user intent rather than code against a written specification. It runs at three levels, system, acceptance, and production, each checked against a different source of intent and signed off by a different person.
Verification asks whether the system was built according to its specification, a question answerable entirely from inside the repository by comparing code to spec. Validation asks whether the system does what the user actually needs, which the repository alone cannot answer, because the specification itself might have been wrong. A build can pass every verification check and still fail validation. For the full validation testing vs verification testing comparison, see the dedicated verification vs validation page at getautonoma.com/blog/verification-vs-validation.
The test oracle problem is that every test has two separable halves, a procedure that drives the system into a state, and an oracle that decides whether that state is the correct one. Automated test generation has made the procedure cheap, but it cannot originate the oracle, because correctness against user intent isn't a property the code exposes. A test can execute a flow perfectly and still assert nothing about whether the result was right.
Validation happens at three levels. System-level validation checks the fully assembled application against its requirements and design intent. Acceptance-level validation checks one feature or user story against written acceptance criteria, signed off by a product owner or customer. Production-level validation checks the shipped system against real user behavior, using signals like usage data, support tickets, and beta feedback that no pre-release environment can produce.
Autonoma handles the procedure half of validation: its agents read the codebase, drive the running application into the states a validation check needs to examine, and re-derive that procedure automatically as the code changes through the Diffs Agent. It does not originate the oracle for user intent. Autonoma surfaces the behavior and the diff; deciding whether that behavior is what the product was supposed to do stays with the person who owns the acceptance criteria, the beta program, or the release.
