Test entry and exit criteria are the conditions that must hold before a test cycle starts (entry) and before a release ships (exit). A criterion only works as a gate if it can fail informatively: a stated number, not a status report like "all test cases executed," which is true or false but never tells you whether the release is actually safe. Measurable exit criteria name a threshold, plus what goes wrong if that threshold is set too loose and what goes wrong if it's set too tight.
Open the testing section of nearly every strategy template and "exit criteria" is the one heading with nothing under it. Maybe a placeholder line: "all test cases executed, no open critical defects." That sentence has never blocked a release, because it can't fail in a way anyone can act on. Either every test ran or it didn't, and "critical" gets defined by whoever argues loudest in the room that day. If you're filling in a test plan template right now, this is the section that keeps coming back blank.
This isn't a document for the team standing up its first suite, and it isn't for the team wondering whether its AI-generated tests are trustworthy at all; those are real problems with their own answers elsewhere. This is for the QA lead, QA manager, or engineering manager who owns quality without a dedicated QA function, somewhere between Series A and Series C, with a real suite and a real release cadence. Somebody upstream asked for a document, and the exit criteria section is supposed to say when it's safe to ship. Right now it says nothing.
A test strategy allocates a scarce resource. For most of the industry's history that resource was time to write tests. It is now attention to review them, because a generator produces test cases faster than any release cadence can absorb, and the exit criteria section is where that shows up first: it has to say which tests earn the right to block a release, not just that they ran.
Why most exit criteria in testing are not gates
A gate is only a gate if it can fail informatively. Tell someone a release is blocked because "not all test cases executed" and the next question is always the same: which ones, and does it matter? The criterion has no answer, because it was never measuring anything except whether a runner finished its list.
Compare that to "zero open P1 defects." If it fails, you know exactly what's blocking the release and who owns the fix. A status report tells you what happened; a gate tells you what happens next when the number is wrong. "All high-severity defects closed" fails the same test as long as "high-severity" gets defined at the moment of the argument instead of in the document itself.
Exit criteria worth writing
Five exit criteria hold up as gates because each one names a number, and each number has a stated failure mode in both directions:
| Criterion | Starting threshold | Too loose means | Too tight means |
|---|---|---|---|
| Open defect count | 0 open P1, P2 below a ceiling | P1s ship, severity stops meaning anything | Cosmetic P2 backlog blocks every release |
| Pass rate per suite tier | Smoke 100%, regression 98%, nightly 95% | One flaky smoke test blocks nothing | A flaky test in one tier blocks all |
| Flake-rate ceiling | Under 2% of runs | A green run proves nothing at all | Real regressions get dismissed as flaky |
| Coverage of changed lines | 80% of lines changed this PR | New code ships with zero verification | Trivial changes forced into full authoring |
| Max age of full-suite run | Under 24 hours old | Gate certifies a build nobody retested | Full suite reruns block every release |
Open defect count earns its bluntness for P1s: zero is the only number that keeps "critical" meaning something. P2 needs a stated ceiling rather than "as few as possible," which gets argued down under pressure.
Pass rate per suite tier exists because one global number hides which tier is failing. Risk-based testing already tells you which tier carries more consequence, so that tier gets the tighter bar.
The flake-rate ceiling is the criterion most templates skip, and it voids every other number without it: once flake crosses a real ceiling, a green run stops being evidence of anything. The fix is the suite's reliability, not the threshold. Google's own infrastructure team reported flaky results in about 1.5% of all test runs, and roughly 84% of pass-to-fail transitions involved a flaky test rather than a real regression. That's a team with enormous investment in test infrastructure sitting near 1.5%, so a ceiling set far above 2% for a team with less of that investment is measuring mostly noise.
Coverage of changed lines, not the codebase, survives contact with a real release; a whole-codebase percentage rewards code nobody touched this cycle. Assertion coverage versus line coverage and a QA metrics dashboard belong in the broader document, but here what matters is whether the changed lines have any test touching them.
Max age of the last full-suite run exists because a fast, narrow gate can stay green for weeks while the slower suite that would catch a slow-building regression hasn't run in longer than anyone noticed.
None of these five numbers are industry standards; presenting them that way would undercut the exercise. They're defensible starting points, adjustable to your risk posture, and reasonable only if the reasoning behind each is written down.
The flake-rate ceiling sits above the other four criteria rather than beside them: once it breaks, the four green checks below it stop being evidence, and the gate is blocked regardless of what they say.
Test entry criteria nobody writes
Exit criteria at least get a heading, even an empty one. Entry criteria usually don't get mentioned at all, which is strange: a cycle that starts on bad footing produces results nobody should trust.
| Entry criterion | Starting threshold | Too loose means | Too tight means |
|---|---|---|---|
| Environment parity | Data and integrations match prod | Prod-only failures slip through | Env too costly to run often |
| Test data availability | Data seeded for each scenario | Tests run on whatever data exists | Hand-built data every time |
| Feature-flag state known | Flag state recorded per run | Flags drift, can't reproduce failures | Every flag frozen before runs |
Environment parity is the first one: if the test environment's data volume, integrations, or infrastructure diverge meaningfully from production, a passing suite proves the software works somewhere nobody's customers do.
Test data availability is the second: a cycle that starts without the specific data its scenarios need, an account mid-trial, a cart with an expired promo code, is testing whatever data happened to be sitting there, not the scenario itself.
Feature-flag state known is the third: a run against an unknown mix of flags is testing an ambiguous blend no real user sees, and the criterion isn't "all flags off," just that the state for the run is recorded and intentional.
Entry criteria and exit criteria are the same kind of object at opposite ends of the cycle: three numbers that have to hold before the work starts, and three that have to hold before it ships.
How Autonoma makes a behavioral gate practical
Every criterion above assumes somebody is producing enough behavioral test coverage, on the surface that changed, to make the gate meaningful. That assumption used to be the actual bottleneck. A gate phrased as "these user journeys pass on the changed surface" is the right gate, since it fails informatively and can't be satisfied by a coverage number alone. It was also, historically, impractical to write, because writing enough behavioral end-to-end coverage to gate a release on was bounded by how many engineer-hours a team had free to author and maintain it. Teams settled for the checkbox versions instead, "all tests passed," "90% coverage," because those were the versions somebody had time to produce.
That's the constraint we built Autonoma to remove. Connect a codebase and our Planner agent reads the routes, components, and user flows to derive the behavioral test cases a release needs, including the endpoints required to put the database in the right state for each scenario. Our Diffs Agent keeps that suite aligned with the changed surface on every PR, adding and retiring tests as flows change, so "coverage of changed lines" stops being aspirational: tests for what changed exist because they were generated for it. The per-tier pass-rate gate gets the same benefit, since every tier needs tests produced fast enough to keep it populated, which is the job our agents do continuously instead of in a quarterly sprint.
None of this replaces the other criteria in this document. Autonoma doesn't touch open defect counts, doesn't run your unit suite, and has nothing to say about performance or load thresholds in your strategy. What it changes is whether the behavioral coverage feeding your pass-rate and changed-line gates can exist at the pace your release cadence demands, not the pace one team had time to hand-write it.
Why coverage percentage is a broken gate for generated suites
"90% line coverage" reads like a safe, objective exit criterion until the suite generating those lines is a machine that will hit any number you name. A human chasing an easy 90% could write shallow tests touching every branch without asserting real behavior, but doing that by hand was slow enough that most teams didn't bother, so the gate happened to work in practice even though it was never sound in theory.
A generator removes that friction. Ask any AI test-authoring tool, ours included, to hit a coverage number, and it will produce tests that execute every targeted line and assert almost nothing about correctness. The number stops meaning "this is protected" and starts meaning "a generator was told to hit this number." Our companion post on why coverage misleads for AI-generated tests goes deeper on the mechanism.
That's why "coverage of changed lines" earns a place above and a raw percentage doesn't: the gate has to be behavioral, or a generated suite will satisfy it without adding real protection. Test automation metrics that predict release quality belong on the same dashboard for a different reason: they measure whether the gate worked after the fact, not before the release.
Google's own testing team frames this exact tradeoff as gating on coverage for all code versus gating on coverage to new code only, and lands on the same side this article does: cover what changed, not the whole codebase. Their own per-commit thresholds run higher, from 90% up to 99%, than the 80% starting point above, which only underscores that the number itself is a risk-posture call, while the choice of gate, changed code over the whole codebase, is the part worth copying outright.
Write the thresholds down before the release meeting does it for you
Test entry and exit criteria, including the eight above, are a first draft with the reasoning attached, the one thing every template out there was missing. Adjust the P2 ceiling for how much support load your team can absorb, the flake-rate ceiling for how new your suite is, and the changed-line coverage number once your test generation can keep up with it. What doesn't change is that whatever number you choose has to fail informatively, in either direction, or it was never a gate.
The alternative is the one you probably already have: a release meeting where the honest answer depends on who's in the room and how tired everyone is. Writing the threshold down first, with its failure modes stated next to it, turns that meeting into a five-minute check instead of a negotiation. If the behavioral suite behind your pass-rate and coverage gates is generated and kept current by Autonoma rather than hand-authored on a schedule nobody controls, that check gets easier every release, because the tests for what changed already exist by the time the meeting starts.
Frequently Asked Questions
Entry criteria are the conditions that must be true before a test cycle can start: the environment matches production closely enough to trust the results, the test data needed for the scenarios exists, and the state of any feature flags is known rather than assumed. Exit criteria are the conditions that must be true before a release ships, typically a defect count ceiling, a pass rate by suite tier, and a maximum flake rate. Both only function as gates if they're written as numbers that can fail informatively, not as status descriptions like "testing is complete."
A good exit criterion names a number, states what happens if that number isn't met, and can be checked without a debate. "Zero open P1 defects" qualifies. "All test cases executed" doesn't, because it never tells you whether the release is actually safe, only whether a test runner finished its list. The five criteria in this article, a defect ceiling, per-tier pass rate, a flake-rate ceiling, coverage of changed lines, and a maximum age for the last full-suite run, are starting points with their failure modes shown, meant to be adjusted to your own risk posture rather than adopted as written.
There's no single number that fits every team, but a per-tier structure works better than one global figure: something like 100% on the smoke tier that runs on every commit, a slightly lower bar like 98% on the broader regression tier, and a bar like 95% on a full nightly run that covers the long tail. A single global pass rate hides which tier is actually failing, and a tier that carries more release risk should carry a tighter bar than one that doesn't.
Raw line or statement coverage across the whole codebase is a weak exit criterion because a test suite, especially a generated one, can hit any coverage number without asserting anything meaningful about behavior. Coverage of the lines changed in the current release is a stronger, narrower version of the same idea: it asks whether the new code has any protection at all, rather than whether the codebase as a whole is superficially exercised. Neither replaces a behavioral check on what the change was actually supposed to do.
That has to be named in the document itself, not decided in the release meeting under deadline pressure. Most teams that get this right name a single accountable role, often the engineering lead or QA lead who owns the release, require a written reason for the exception, and set an expiration on it rather than a standing waiver. An exit criterion that anyone in the room can waive under deadline pressure was never really a gate.
Suspension criteria are the numeric trigger that halts a test cycle instead of letting it keep producing results nobody trusts, for example a smoke-tier pass rate dropping below its own threshold or a shared environment going down entirely. Resumption criteria are the numeric condition that has to be true again before testing restarts, such as the blocking defect being fixed and the environment passing its own health check. Like every other gate in this article, both only work if they're written as a threshold that can fail informatively, not as "pause if things look bad."
Autonoma's Planner and Diffs Agent generate and maintain the end-to-end behavioral tests that a pass-rate or changed-line coverage gate needs to be meaningful, running them against a live preview environment and keeping the suite aligned with the changed surface on every PR. It only covers the behavioral E2E layer, though: it doesn't run your unit tests, measure performance or load, or replace an accessibility scanner, so those parts of your exit criteria still need their own tool or owner.
