What Is Sanity Testing? Checking the Fix That Shipped

You just merged a one-line fix. A discount rule that should have applied at exactly $100 in cart value was, until ten minutes ago, refusing to fire until $100.01. The change is trivial: a > becomes a >=. Whether the line compiles was never the question. What actually matters is what you check next, and how far out from that one line you look before calling it done. This is for the engineer running that check right now, not the person compiling next quarter's testing strategy.

What sanity testing actually checks

Sanity testing runs immediately after a targeted, usually small, code change, and its job is narrow on purpose: confirm the specific thing that changed behaves correctly, and confirm the handful of behaviors closest to it still work too. Not the whole application. Not even the whole feature the change lives inside. Just the changed rule and its immediate neighbors.

That scope is what separates sanity from the gate it gets confused with most. Smoke testing is wide and shallow: it runs on every new build and asks whether the product is alive at all, does it launch, does the homepage load, can a user log in. Sanity testing is narrow and deep: it runs only after a targeted fix and asks whether that fix, specifically, works. For the full three-way breakdown against regression, plus a decision flow for which gate fires on which trigger, see Smoke vs Sanity vs Regression Testing. For where sanity sits inside the wider release vocabulary, the software testing terminology guide covers the full set of terms this article assumes.

Who this is for

This is written for the engineer, SDET, or release owner who just shipped a fix and has to decide what to check before it advances, staring at a merged pull request and wondering how far out from the change to look. It is not written for the QA lead deciding how a team allocates its whole suite across a sprint, that's a strategy document, not a five-minute check. It is not written for a team asking whether its AI-generated tests are real, that question belongs to AI test theater. And it is not written for someone choosing input values for a single test case, boundary values, partitions, that's one level down, covered in test design.

A sanity testing example: the $100 discount boundary

Here's a fix worth tracing end to end. A checkout page applies a 15% discount to any order with a subtotal of $100 or more. The eligibility rule read subtotal > 100, not subtotal >= 100, so a cart that totaled exactly $100.00 was denied a discount it was supposed to get. The fix is one character: > becomes >=.

That one-character change is exactly the kind of fix sanity testing exists for: too small to justify a full regression pass, too easy to get subtly wrong to ship unchecked. A boundary rewritten from strict to inclusive can just as easily be mistyped into applying the discount a dollar too early. So the check runs several inputs through the changed branch, not one happy-path case:

Case	Input	Expected	Why in scope
Below threshold	Subtotal $99.99	No discount, total $99.99	Confirms the boundary wasn't loosened
Exact boundary	Subtotal $100.00	15% off, total $85.00	The failure the fix targets
Just above boundary	Subtotal $100.01	15% off, total $85.01	Confirms the working case stayed intact
Well above threshold	Subtotal $250.00	15% off, total $212.50	General path still correct
Checkout summary	Cart totals $100.00	Summary shows $85.00, not $100.00	Consumes the rule's output
Invoice record	Order placed at $100.00	Invoice records $85.00 total	Second caller of the same rule

The fix changes exactly one verdict on this line: the $100 boundary. That is where the sanity check aims.

Row two is the case the fix exists for. Rows one, three, and four aren't padding, they're what confirms the fix didn't overcorrect. Rows five and six are a different kind of case entirely: they aren't testing the rule again, they're testing who consumes its output.

The blast radius: what else the fix could break

A discount rule rarely lives alone. In this codebase, the eligibility function is called from two other places: the checkout page, which reads its return value to render the total the customer sees before paying, and the order-confirmation flow, which calls it again when the invoice record gets written after payment succeeds. Those two callers are the immediate blast radius, rows five and six in the table above, and they matter for a specific failure mode boundary fixes are prone to: the fix works exactly where you tested it, and something else reading the same output is now inconsistent with it.

If the checkout total shows $85.00 but the invoice records $100.00, because the invoice path caches an earlier value or calls a stale copy of the same rule, the customer walks away with a receipt that doesn't match what they paid. That's a narrow defect, but a real one, and it's invisible if the sanity check only re-runs the rule and never asks who reads its output.

The rule and its two callers are the sanity scope; everything in the outer box is a regression question.

The blast radius isn't "everything downstream of checkout." It's specifically the callers of the function that changed, found by asking who calls this, not by asking what could possibly be affected somewhere. That distinction is what keeps a sanity check fast: minutes, not a release cycle.

How to do a sanity test

Start by reading the diff, not the ticket, and name exactly what changed, down to the function. Then ask who calls that function, and take the answer from the codebase, not memory, because those callers are the blast radius. Run the changed behavior across the values on either side of whatever the change moved, not just the one case that used to fail. Confirm each caller still reflects the corrected output, not just the function in isolation. If the check takes longer than a few minutes, the scope has already grown past sanity into regression.

What sanity deliberately doesn't cover

Sanity testing is narrow by design, not as a shortcut but as its whole point. The worked check above validated four boundary cases on one function and two consumers of that function's output. It said nothing about whether the promo banner mid-cart still shows the right badge, whether the loyalty-points calculation that runs after checkout still awards the right amount, or whether an unrelated change merged last week interacts badly with this one.

Those questions belong to regression testing: the broader, planned pass that runs before a release to confirm nothing else moved. A team that runs only sanity checks before a release is skipping regression, not doing a smaller version of it. Retesting is the third related term worth pinning down: retesting reruns the exact case that failed, sanity checks the fix plus its neighbors, regression checks everything else.

Retest checks one case, sanity checks the fix and its callers, regression checks everything else.

How Autonoma scopes a sanity check from the change

Every case in the worked example above lived inside one function's blast radius: the rule itself, the checkout total that reads its output, the invoice that reads it a second time. Finding that blast radius by hand means opening the codebase, searching for every call site of the changed function, and trusting the search caught everything, a step teams skip under a deploy deadline more often than anyone admits. The two behaviors closest to a fix are exactly the ones most likely to break quietly.

That's the concrete version of the grey-box move. Our Diffs Agent reads the pull request that produced the fix, not just the changed lines but the function and call sites those lines belong to, and derives which behaviors sit in the immediate blast radius from that reading rather than from a list someone maintains. It then verifies the derived scope by driving the actual, running application in a preview environment, the same way a person would click through checkout and pull up the invoice, except the scope it checks is read from the change itself. Autonoma doesn't decide whether the rule itself is correct, that's still yours, but it does the reachability search a person is most likely to skip under deadline.

Map that back to the worked example: the diff is the one-character comparison fix; the changed surface is the eligibility function and its two call sites; the derived sanity scope is the four boundary cases on the rule plus the two blast-radius checks on checkout total and invoice record, the same six rows in the table above, generated and run without anyone re-typing them by hand.

Sanity scope is derivable from the diff

Every case in the worked example, the four boundary cases and the two blast-radius checks, came from one place: reading the diff and asking what calls this. That's worth stating plainly, because most teams don't treat sanity that way. A "sanity suite" is usually a document someone wrote once, a checklist tab that gets updated when a person remembers to: log in, view cart, apply a coupon, done. The list outlives the code it was written for. Six months later, half the entries test a flow that changed shape, and the flow that actually needs checking today isn't on the list at all.

This gate was a curated, hand-maintained list when a human had to read the diff, guess at the blast radius, and write the check by hand every time. When the checks are instead derived from the codebase itself, the diff, the changed function, its call sites, and healed automatically as those call sites change, the gate stops being a static list anyone owns and becomes a signal you regenerate on demand. The scarce work moves from writing the six rows in the table above to reviewing what they surface, which is the one step in this whole exercise that still needs a person.

The diff names the function, the function names its callers, and the callers are the scope.

One diff, one scope, not a whole release

The fix in the example above was one comparison operator. Most sanity checks are exactly that small: a targeted change, a handful of boundary cases on the changed branch, two or three neighbors that consume its output. The discipline that actually matters is resisting two opposite urges, skipping the check because the fix "looked safe," or running the whole regression suite because scoping down feels risky. Neither serves the moment. The diff already tells you what changed; the sanity scope is just an honest reading of it.

If you'd rather have that reading done for you than reconstruct it from memory every time a fix ships, that's the specific job Autonoma does: it reads the diff you just merged, derives the scope, and runs it against your preview environment before you've finished writing the pull request description.

Frequently Asked Questions