Sanity vs Smoke Testing vs Regression: 3 Triggers

Every article on smoke vs sanity testing hands you a two-column table and a period at the end of the sentence. None of them tells you what to do the moment a build turns green, a fix lands, or a release window opens. That's the question that sent you here, answered below with a decision flow that names the trigger for each gate, not just another table.

This is for whoever is standing at that fork right now: the engineer watching a pipeline go green, the SDET who owns the release gate, the release manager deciding whether a build ships. It is not for the QA lead building a long-term suite-allocation strategy, and not for the reader asking whether an AI-generated suite tests anything real; those are answered elsewhere. Picking the inputs for one test case, a boundary value here, an equivalence class there, is one layer further down still. This page has one job: given the event that just fired, which gate do you run?

Smoke vs sanity vs regression: what each gate checks

Smoke, sanity and regression aren't three names for the same check run at different times. They check different things, on different triggers, at different depths, and mixing them up wastes either pipeline minutes or nerve.

Smoke testing asks one question: does the build come up? Log in, load the homepage, hit the core path, stop there. It's the shallowest and widest of the three, runs first, in seconds, and a failure means stop: don't run anything else against a build that can't clear the front door. That instinct isn't new: continuous integration has always depended on a fast first stage, what Martin Fowler calls the commit build, which he says should take only a few minutes. Smoke testing covers the full definition and a worked example suite, and a worked set of smoke test cases shows what that shallow pass actually contains.

Sanity testing asks a narrower question about one place: does the area a fix just touched behave as intended? It runs after a targeted change, in minutes, deep but narrow, and a failure means the fix is wrong or incomplete, not that the rest of the app broke too. Sanity testing covers the trigger-and-diff framing in full.

Regression testing asks the broadest question: did anything else break? It runs before a release or after a large merge, slowest and widest by design, because it covers ground the other two gates were never scoped to touch. A failure means a change had blast radius outside the surface it was meant to touch. Regression testing is the full guide; the automated regression testing guide covers what keeps that scope affordable on every merge.

Smoke testing vs sanity testing vs regression, across 5 axes

Here's the comparison most write-ups give you, minus the part where it stops at definitions.

	Smoke	Sanity	Regression
Scope	Widest, shallowest	Narrowest, deepest	Broadest, still deep
Trigger	New build	Targeted fix	Pre-release, large merge
Depth	One pass per core flow	Every path through changed area	Every path across the app
Cadence	Every build, seconds	After each fix, minutes	Before release, longest run
A failure means	Stop, build isn't testable	Fix is wrong or incomplete	Change had blast radius

Smoke test vs regression test, the pair that looks most alike

Both smoke and regression run wide across the whole app, which is exactly why people confuse them. What separates them is depth and trigger, not breadth: smoke goes one pass deep on every core flow and fires on a build, while regression goes every path deep and fires before a release.

Every axis points the same direction: smoke is fastest and shallowest, sanity is narrowest and most surgical, regression is slowest and broadest because it's the only gate checking for damage nobody predicted. None is more important than the others; they answer three different questions at three different moments, and running the wrong one at the right moment is how a bad build reaches production or a good fix gets stuck behind an hour of unrelated checks.

Which test to run when: each trigger and the gate it fires

The table above is honest, but it still leaves you doing the mapping yourself. Here's that mapping, stated directly, the thing most write-ups leave you to work out yourself:

The gate is a function of the trigger, not a fixed suite picked in advance.

Notice what actually decides the gate: not the size of the change, not how nervous it makes you, just the trigger. A build went green: run smoke. A fix landed for one bug: run sanity. A release window opened, or a large merge just landed: run regression. If you're running the wrong gate for the trigger in front of you today, that's the fix.

Scope versus depth: where smoke, sanity and regression sit

The three gates also separate along two dimensions at once: how much of the app they touch, and how deep they go into what they touch. Plotting that makes the shape of each gate visible in a way the table alone doesn't:

Smoke trades depth for width, sanity trades width for depth, and regression refuses to trade at all.

Regression sits in the corner that costs the most to run, because it's the only gate that doesn't get to specialize.

It has to be wide because a change anywhere can be the one that broke, and deep because a shallow pass would miss what a shallow smoke check is designed to accept. Smoke and sanity earn their speed by trading something away, width in sanity's case, depth in smoke's; regression never gets to make that trade, which is why the regression testing guide spends far more time on it than a comparison page needs to. The cost is real outside any one codebase, too: Google's own engineering team has written about being unable to regression test each code change individually, which produces exactly the kind of lag this section is describing, between a change landing and its test result coming back.

How Autonoma runs one suite at three scopes

Every gate above assumes a person curated it. Someone wrote the shallow login-and-load check that becomes the smoke suite, someone else wrote the narrow, deep pack scoped to checkout that becomes a sanity check, and a third person maintains the broad, slow suite that runs before release. Written that way, there are three suites, three owners, and three places for drift to creep in the moment the app changes and nobody updates all three.

We built Autonoma's agents to read the codebase directly instead of working from three separately maintained lists. Our agents derive test cases from the routes, components and flows that make up your app, the way a person would trace through the code to write a suite by hand, and our Diffs Agent keeps that derived suite current on every pull request by reading the code diff itself.

Because the suite comes from the code rather than from a person's memory of what each gate is supposed to cover, the same suite can run at three scopes depending on what triggered it: a build event runs the widest, shallowest slice as the smoke check; a diff-scoped event runs the deepest, narrowest slice over the changed area as the sanity check; a pre-release event runs the whole thing as the regression check.

That's not a claim that the three moments stop being distinct events; a build going green is still a different moment than a release window opening, and each still deserves its own pass or fail signal on its own timeline. What changes is what sits behind each signal: not three suites written by three people at three different times, but one regenerated, self-healing set of checks, sliced to a different scope depending on which trigger fired. One suite, three triggers, nothing left to quietly fall out of sync with the code underneath it.

The suite in the middle never changes; the trigger just picks which slice of it runs.

Where the split still matters, and where it stops

The habit of treating smoke, sanity and regression as three fixed, hand-authored suites was solid for the era that produced it. A person at a keyboard could only afford to write and run so many checks, so splitting the work into three different-sized suites, one cheap and shallow, one narrow and targeted, one comprehensive and slow, was the only realistic way to get reasonable coverage without rerunning everything on every change.

That's the assumption a gate like this used to run on: a curated, hand-maintained set of checks that a person kept updating by hand, run only when someone judged it worth their time. Once the checks behind a gate are derived from the codebase instead, and heal themselves automatically when the interface they check moves, the gate stops being a fixed suite someone maintains and becomes a signal you regenerate on demand, at whatever scope the trigger calls for. The scarce resource shifts too: it's no longer authoring or maintaining the checks, it's reviewing what a run actually surfaces.

If you're deciding whether this build ships right now, none of that changes what you do next: the trigger still tells you the gate. The re-architecture matters if you're asking why maintaining three separate suites used to cost what it cost, not if you're mid-incident deciding what to run next.

Where the split still matters is the trigger itself and the promise attached to it. A build going green still isn't the same event as a release window opening, and treating them as interchangeable, running the full suite on every commit or skipping straight to release with no smoke check, wastes either pipeline minutes or nerve. The three triggers aren't going anywhere. What stops holding is the assumption that a separately maintained suite has to sit behind each one.

What no gate here covers

None of the three gates above, however they're generated, cover every kind of testing a release needs. Exploratory testing, a person deliberately trying to break the app in ways no script anticipated, stays a human skill; no suite replaces someone actively hunting for the unexpected. A real-user beta catches failure modes only real devices and habits surface. Load and performance testing is a different discipline, concerned with behavior under volume, not whether one flow works once. Accessibility testing needs its own tools, ideally with people who rely on assistive technology testing the result directly.

Autonoma's agents sit inside this picture as the behavioral, end-to-end execution layer behind smoke, sanity and regression, not a replacement for the unit tests underneath them, the load rig running on its own schedule, or the accessibility scanner checking markup none of this touches. Smoke, sanity and regression tell you whether the application behaves the way the code says it should, not whether it behaves well for someone using a screen reader or holds up under heavy concurrent load, and no amount of codebase-derived test generation changes that boundary.

Autonoma runs the three code-derived gates; exploratory, load, and accessibility stay with people and dedicated tools.

Where regression testing goes deeper than this page can

Regression is the gate this page has the least room to do justice to, on purpose. It's the broadest of the three, and it's already covered by other posts on this site that go further than a comparison page needs to. The regression testing guide covers the full definition, the different strategies, and when each is worth the cost. The automated regression testing guide goes further still, into what keeps a regression suite affordable enough to run on every merge instead of once a quarter. If regression is the gate you landed here to understand in depth, start there.

None of that changes the question this page set out to answer. The next time a build goes green, a fix lands, or a release window opens, the gate is already decided: match it to the trigger, not to how big the change feels. That's true whether the suite behind each gate is hand-written and hand-maintained, or, like the one behind Autonoma, regenerated from the code itself every time it runs.

Frequently Asked Questions