What Does Smoke Testing Actually Check?

Ten minutes after a deploy, the checkout endpoint starts returning errors. Nobody notices until QA has already spent an hour running the full regression suite against a build that was never going to ship. That wasted hour comes down to one missing answer: what a fast, shallow check should have looked at before anyone touched the full suite at all.

The answer isn't a checklist of ten sacred test cases picked once and left alone. Smoke testing checks whether every critical subsystem, login, the primary write path, the primary read path, payment, responds at all, and it does that in minutes rather than the hour a full suite costs, before anyone decides whether the build deserves anything deeper. What separates a real check from a vague checklist item comes down to four specific properties, not a length or a number of test cases.

This is written for the engineer, SDET, or release owner staring at a fresh build and deciding whether it earns the next two hours of testing, or the person who just got told to name and run that gate. It is not written for someone designing a suite-allocation strategy across a QA org, someone asking whether an AI-generated test suite is actually testing anything, or someone choosing edge-case inputs for a single test case. Those are real questions. They live one level up or one level down from this one.

What Is Smoke Testing in Software Testing?

Smoke Testing Meaning: Where the Name Comes From

Smoke testing gets its name from hardware. After assembling or repairing a circuit board, an engineer would power it on and watch for smoke: none meant the board was safe enough to test further, and any meant stop, because there was no point running instruments against a board that was already failing. Software testing borrowed the discipline and the name, a practice Steve McConnell documented as the "daily build and smoke test" in IEEE Software back in 1996, along with two more formal aliases that show up in job postings and process docs: build verification testing and build acceptance testing. All three names describe the same gate.

If the terms across build stages still blur together, the software testing terminology guide sorts smoke and its neighbors into one place.

The Four Properties That Define the Job

Four properties define the job, and none of them is a length. Shallow means one level deep: does the login screen accept valid credentials, not what happens when the password field gets a twelve-character string with an emoji in it. Wide means every critical subsystem gets touched once: authentication, the primary write path, the primary read path, payment, whatever the product cannot function without. Run-first means before anything expensive: no unit suite, no regression corpus, and no manual QA pass gets scheduled time until smoke has cleared. Decision-making is the property most category-label definitions skip entirely: a smoke test doesn't return a bug count or a coverage percentage, it returns one answer, go or no-go on whether the rest of testing is worth starting today.

Smoke goes wide and stops at one level; regression trades width for four levels of depth.

That shape, wide and shallow, is also why the audience for this page stays narrow. Deciding whether today's build clears the gate, or owning that decision for a team, is what the rest of this article answers. Allocating a QA team's time across a broader suite strategy, or checking whether an AI-generated suite is actually asserting anything real, is a different problem with its own page.

One Worked Smoke Suite Example: An E-Commerce Storefront

Smoke isn't the only gate that runs at this stage, and it isn't the only one worth naming precisely. If you're trying to decide whether a given moment calls for smoke, sanity, or a full regression pass, the three-way comparison and decision flow are worth reading before you standardize on one; and if the check-after-a-fix is what you actually need, sanity testing is its own gate with its own definition. This page owns exactly one thing: what a smoke suite looks like when you write it down.

Take a concrete shape: an e-commerce storefront, a web application with accounts and checkout. A smoke suite for it needs at least four checks, one per critical subsystem, and every action has to be specific enough that someone, or something, could actually run it. Not "check login works." This:

Check	Pre-condition	Action	Expected result	Actual result
Authentication	Seeded account exists in the database	POST /api/sessions with seeded credentials	200 response, session cookie set	200 response, session cookie set
Core create	SKU-1042 is in stock	Click Add to cart on SKU-1042, open /cart	/cart shows 1 item, subtotal updates	/cart shows 1 item, subtotal updates
Core read	Catalog seeded with 50 products	GET /api/products?category=shoes	200 response, 12 shoe results	200 response, 0 results returned
Checkout	Cart has 1 item, valid test card	POST /api/checkout with test card token	201 response, order confirmed	201 response, order confirmed

Every one of these follows the happy path on purpose. Smoke isn't where you test the edges of a form; it's where you confirm the form exists and responds, which is a different and much cheaper question.

Three of the four rows match. The fourth doesn't, and that mismatch is the entire point of the actual-result column. The category filter is returning zero results against an expectation of twelve. That's not a cosmetic issue. If a customer can't browse by category, the storefront's primary read path is broken, and running the rest of the suite (which would take the better part of an hour) against this build would tell you nothing you don't already know: it isn't ready. The gate fails, the pipeline stops, and the fix ships before anyone spends the hour.

Why these four checks and not, say, ten? Because four is what a wide, shallow gate against a single concrete product shape needs to prove the build isn't dead. A real suite for a production application usually runs longer than four rows; a full set of copyable smoke test cases, plus a reusable checklist for building your own, covers that longer version. This page shows the one worked example, not the whole set.

The read path returns zero where twelve were expected, and that one mismatch fails the whole gate.

Smoke as a Property of the Build, Not a List You Prune

For as long as smoke testing has existed, a smoke suite has been a curated list, usually somewhere around ten critical paths, because a human had to run each one by hand and ten is roughly what fits before someone gives up and ships anyway. That number wasn't arbitrary, but it also wasn't the important part.

The real content of a smoke suite was never the ten cases. It was the criteria used to pick them: which paths carry the most traffic, which ones block revenue if they break, which ones are unrecoverable rather than merely annoying if they fail. Someone sat down, looked at the product, and made those calls once. Everyone else maintained the resulting list by hand for years afterward, updating it every time a route moved, a button got renamed, or a new critical flow shipped without anyone remembering to add it.

A pass releases the expensive stages; a fail routes straight back to the developer.

Those criteria, highest-traffic, revenue-blocking, unrecoverable-if-broken, are exactly the input a generator needs. All three are things you can determine by reading the codebase: which routes get the most calls, which ones touch payment or the primary write path, which ones have no fallback if they fail. Once the paths are derived from the code instead of remembered by whoever wrote the original list, and once they heal themselves when a selector or a route shifts instead of breaking on the next deploy, the suite stops being a list anyone prunes. It becomes a property of the build itself, regenerated every time the build changes, rather than an artifact someone has to remember to keep current.

That shift also moves where the human time goes. Writing and maintaining the original ten checks used to be the scarce work, the thing that ate a sprint whenever the product changed shape. When the checks regenerate themselves on every build, the scarce work becomes reviewing the one that failed, not writing the other nine that didn't.

None of that makes automating the gate a foregone conclusion. Whether it's worth automating in the first place depends on how often you deploy and how long a suite takes to run by hand; the tipping point is its own arithmetic. And once a suite is automated, wiring a smoke gate into CI so it actually blocks a bad promote is a separate problem.

How Autonoma Builds a Smoke Suite From Your Codebase

The pattern this article just described, checks derived from criteria instead of memorized as a list, only works if something can actually read the codebase and apply those criteria consistently, run after run, deploy after deploy. That's the specific gap between "smoke testing should be a property of the build" as an idea and a gate a team can rely on every day without someone quietly maintaining it by hand.

Autonoma is built around that gap. Our agents read the codebase directly, find the routes and flows that actually exist and matter (the login form, the primary create and read paths, the checkout flow), and generate checks against them the same way a human would if they had time to re-derive the criteria on every single build. Those checks then run against the live, deployed preview of the application rather than against a saved snapshot of the DOM, which is what lets a check survive a UI change instead of breaking the moment a class name or a layout shifts. On every pull request, the Diffs Agent re-reads what changed and updates the suite accordingly: paths that no longer exist get dropped, new critical paths get picked up, and nobody has to remember to do either by hand.

Autonoma derives the suite from the code and the Diffs Agent regenerates it on every pull request.

That's a specific kind of coverage, not the whole map. A tool built to drive the running application complements a unit runner, a contract-testing framework, a load-testing tool, an accessibility scanner, and a human QA hire, and was never meant to replace any of them. If the gap is unit-level structural coverage or a contract between two services, or a load test or an accessibility audit, those tools stay exactly where they are; what changes is that the smoke layer stops being the one piece of that stack someone has to remember to update by hand.

Mapped onto the worked suite above, that means the four rows aren't something a person wrote once and now maintains forever. They're regenerated from whatever the storefront's codebase actually contains this week, which is the whole argument this article has been making, made concrete.

What Smoke Testing Won't Catch

A smoke gate is deliberately shallow, and being honest about that is part of using it correctly. It will not find edge cases, like what happens at the boundary of a discount code, or with a username at the character limit. It will not find data-boundary bugs, gaps in a permissions matrix between user roles, race conditions under concurrent load, performance regressions, or accessibility failures. A green smoke run means the build is worth testing further. It does not mean the build is correct.

Those deeper questions belong to other gates. The broad, pre-release pass that catches regressions across the whole application lives in the regression corpus; the level that tests the fully assembled product as a black box against its requirements is system testing. Smoke is the first gate, not the last one, and treating it as full coverage is how teams end up shipping a build that passed smoke and broke everything else.

Some kinds of testing stay human no matter how good the automation gets, and that's a limit of the gate itself, not of any particular tool. A beta program that puts a build in front of real users, and the exploratory pass where someone pokes at the product because something felt off, are both judgment calls a smoke gate structurally can't make on your behalf. No amount of automating the checks changes that; the gate answers go or no-go, not whether real users will like what shipped.

Where Smoke Sits Next to Sanity, Regression, and System

Smoke answers one question: is this build alive enough to test further. Sanity answers a narrower one after a specific fix: does the change that was just made actually work, without checking everything else. Regression asks the broadest version: did anything, anywhere, break. System testing asks whether the fully assembled product satisfies its requirements, as a black box, independent of how any individual piece was built. Each is a different trigger and a different cadence, not a different quality bar, and the full decision flow for choosing between them, along with the sanity definition on its own, is worth reading if you're setting up more than just the first gate. The same production-facing idea, run against a slice of live traffic instead of a fresh build, is what canary testing covers, and the pipeline mechanics for wiring any of these gates into CI belong to the CI/CD testing guide.

Go back to the storefront example for a second. The four checks that make up its smoke suite aren't sacred, and neither is the number four. If the product added a wishlist feature or switched payment providers, the paths worth checking would change with them. A team that treats its smoke suite as a fixed asset is really treating a moving target as if it held still, which is exactly why so many hand-maintained smoke suites go stale within a year of being written: nobody updates the original ten checks as fast as the product changes shape underneath them.

If you're the one deciding whether today's build clears the first gate, the definition, the worked suite, and the reframe above are the whole job. When you're ready to stop maintaining that list by hand, connecting your repository to Autonoma is what turns it into a gate that regenerates itself instead of one more script someone has to remember to update.

Frequently Asked Questions