System testing is the first level where the fully integrated, assembled product is tested as a black box against its own requirements, with no knowledge of how any individual service or module works internally. It runs after unit and integration testing have already verified the pieces separately, typically against a production-like environment, and it exists to catch a requirement violation that survives even when every pairwise integration test between services passes.
Every integration test in the checkout flow passed. Cart to pricing: passed. Pricing to payment: passed. The pull request merged clean, the pipeline ran green end to end, and a customer got charged ten dollars and eighty cents less than the total they'd agreed to at checkout. Nobody had written a test that would have caught it, because no single service, taken on its own, had done anything wrong.
This is written for the engineer, SDET, or release owner who has to decide whether a fully assembled build satisfies its requirements, or who has just been asked to name and run that check. It is not written for someone building a suite-allocation strategy across a QA organization, someone asking whether an AI-generated test suite is actually asserting anything real, or someone picking the boundary values and equivalence classes for a single test case. Those are real questions. They live one level up or one level down from this one.
System Testing, Defined
System testing in software testing sits at a specific, narrow point in a build's lifecycle: after every individual unit has been checked in isolation, after every pair of services or modules has been checked together, and before the product goes in front of anyone outside engineering. At that point the definition is precise. The product under test is the fully assembled thing, every service, every module, wired together the way it will actually run in production. The test treats it as a black box: nobody writing a system test needs to know that the discount logic lives in its own service, or how any of the internals were built. What matters is only what goes in and what comes out, checked against the product's stated requirements, not against any individual component's contract.
That last distinction, requirements instead of contracts, separates a system test from everything that ran before it. A unit test checks that one function returns the right value for a given input. An integration test checks that two services honor the interface they agreed on. A system test checks something neither of those can see: whether the assembled product, taken as a whole, does what it was supposed to do. It typically runs in a staging environment or another production-like deployment, because a black-box check is only meaningful against something that behaves the way production will.
If the vocabulary across every one of these stages still runs together, the software testing terminology guide sorts smoke, sanity, system, and acceptance into one place.
A system-level check watches only what crosses the outer boundary, never how the services inside reached that answer.
Where System Testing Sits: Unit, Integration, and the Whole Product
Two levels come before system testing, and neither is worth re-arguing here. Unit testing checks a single function or class by itself, usually with everything around it mocked away; the full breakdown from unit through integration to end-to-end is the right page if that boundary itself is what's unclear. Integration testing checks that two or more of those units behave correctly once they're actually wired together, which is also the level most often confused with end-to-end testing, a distinction worth sorting out on its own page.
System testing sits one level above both, and the line between it and integration testing is worth stating precisely rather than gesturing at: integration testing verifies a contract between a pair, or a small cluster, of modules, while system testing verifies the fully assembled product against its own requirements, with no pair singled out. The full boundary between the two, drawn as one diagram, is its own page if you need to place a specific bug precisely.
This is also where the audience for this page narrows again. If you already know which service owns a bug and just need to verify its contract with the one service next to it, you're one level down, in integration testing. If you're deciding whether the assembled, deployed build is allowed to move forward at all, you're exactly where this page is aimed.
Each tier up widens the scope of assembly under test, and only the top tier checks the whole product against its requirements.
A System Testing Example: Priced at $108, Charged $97.20
Take a checkout flow assembled from four services: a cart service that holds line items and computes a subtotal, a pricing service that applies promo codes, an inventory service that reserves stock, and a payment service that charges the card and creates the order record. A customer adds three items totaling $120.00, applies the promo code SAVE10 (ten percent off), and checks out. Three integration tests cover the services that touch pricing directly, and every one of them passes.
| Integration test | What it asserts | Result | Verdict |
|---|---|---|---|
| Cart preview | Preview total after code, no charge | $120.00 to $108.00 | Pass |
| Cart to pricing | Pricing applies code to given amount | $120.00 to $108.00 | Pass |
| Pricing to payment | Charges exactly the amount given | $97.20 charged as given | Pass |
Each one of those is correct, in isolation, on its own terms. The cart service's own preview math is right: $120.00 with ten percent off is $108.00, and that's the number the customer sees on the order summary screen before confirming. The pricing service's contract is right too: given an amount and a code, it applies the discount correctly. And the payment service's contract is the simplest of the three: charge whatever charge_amount says, no more, no less, and it does exactly that.
Here's what none of those three tests can see. When the order is submitted, the order object still carries the promo code as a field, because it travels with the order all the way through the pipeline instead of being consumed once at the cart. The checkout flow calls the pricing service a second time, on the order's total, which is already the discounted $108.00 from the cart's own preview calculation. Ten percent off $108.00 is $97.20, and that's the number that lands in the payment service's charge_amount field. The payment service charges exactly what it's told, faithfully, and the pairwise contract holds all the way down the chain.
The requirement that gets violated only exists at the assembled level: the amount a customer is charged has to equal the total they were shown before they confirmed the order. $108.00 was shown. $97.20 was charged. No single service broke its contract to make that happen. The violation lives entirely in how three individually correct services compose into one flow, which is precisely the class of bug a system-level check is built to catch and a pairwise integration suite structurally cannot.
A system test for this doesn't inspect the pricing service's logic or the payment service's charge_amount field at all. It drives the assembled product the way a customer would: add items, apply the code, read the total on the order summary screen, confirm, then check what was actually charged, at the product boundary, using nothing but what's observable to someone using the product. The assertion is one line: charged amount equals displayed total. That's a check no individual service, and no pairwise contract test between two of them, can make, because it needs both numbers produced by the same flow a real user takes, in the same test run.
Both pairwise contracts pass, and the assembled flow still charges $97.20 against a displayed total of $108.00.
How Autonoma generates system-level tests from your codebase
The gap in the worked example above is exactly the kind of test suite that never gets written and, if it does get written, goes stale fastest. Someone has to know the whole assembled flow, not just their own service's contract, to write the one check that spans cart, pricing, and payment together, and remember to update it every time a field like the promo code changes how it propagates through the order object. A system-level suite used to be a curated, hand-maintained set of end-to-end checks that one person owned and everyone else forgot about, until the day it started failing for reasons nobody on the current team could explain.
That's the specific problem our architecture is built around. Autonoma's agents read the codebase directly, not one service's repository in isolation but the routes, endpoints, and flows that stitch every service into the product a user actually experiences, and derive system-level checks from what that assembled flow does. Those checks then run against a live, deployed preview environment, the same black-box vantage point a system test needs by definition, rather than against a mocked version of any single service. On every pull request, the Diffs Agent re-reads what changed and updates which assembled flows exist and what they should assert, so a check like the one this article walked through stays aligned with how the order object actually moves through the system today, not how it moved six months ago. When checks are derived from the codebase and healed automatically like that, a system-level suite stops being a hand-maintained artifact and becomes a signal you regenerate, and the scarce work becomes reviewing what it surfaces, not authoring and re-authoring the checks themselves.
Mapped onto the order example above: the assertion that a displayed total has to equal a charged amount is exactly the shape of check our agents generate, because it's derived from reading how the checkout flow is actually wired together in the code, not from someone on the payment team remembering to ask the cart team what their preview screen shows.
Autonoma derives the system-level checks from the codebase, and the Diffs Agent regenerates them on every pull request.
Types of System Testing, and What Each One Actually Answers
System testing isn't one check, it's a family, and the family is usually presented as a flat list of labels without saying what question each one actually answers at the assembled-product level. That framing is more useful than the label.
| Type | Question it answers, assembled |
|---|---|
| Functional | Does it do what requirements say, end-to-end? |
| Usability | Can a real user finish the flow? |
| Compatibility | Works across supported browsers and devices? |
| Performance and load | Holds up under realistic concurrent traffic? |
| Security | Can it be made to break access rules? |
| Recovery | Recovers correctly after a crash or outage? |
Not every row belongs to the same kind of check. Performance and load testing needs a tool built to generate and measure traffic at scale, something like k6 or Locust, not a functional check driving one browser session at a time. Accessibility auditing needs a scanner built for the job, like axe-core or Lighthouse, run against the same assembled product. Unit-level structural coverage, whether every branch of the discount logic got exercised, belongs to the language's own unit runner, not to anything running against a deployed environment. A functional system-level check answers the top row directly, plus usability and compatibility when it's run across the right browsers and devices (for a web application specifically, web application testing covers those layer-specific concerns in more depth). The rest belongs to tools built for those jobs, and treating one functional check as coverage of the whole family is how a real gap gets missed.
When System Testing Is the Wrong Level
System testing earns its place at exactly one point in a release: after the pieces have been checked individually and together, and before anything ships. It isn't the right tool for every question that sounds like it belongs at "the whole product" level.
If you already know which service owns a bug and need to verify the one contract at fault, that's integration testing, one level down, and a full system pass to isolate it is slower than it needs to be. If the open question is a different axis entirely, not scope of assembly but what kind of question is being asked of the build, the verification-versus-validation split covers that directly. If you need a real, non-technical person to react to the assembled product, black-box system testing is necessary but not sufficient on its own. Acceptance testing, whether an internal UAT pass or a beta program, is a separate, later gate this article doesn't cover. And if the question is purely load or accessibility, those live in the type breakdown above, not in a generic functional pass.
None of that makes system testing less useful. It makes it exactly what its name says: a test of the system, not one part of it, and not of whether the system was the right thing to build in the first place.
The definition, the pyramid placement, and the order example above are the whole job if you're deciding whether today's assembled build is allowed to move forward. The four services in that example each did exactly what their own tests said they would, and the product still charged the wrong amount, which is the entire argument for testing the assembled thing and not just its parts. When you're ready to stop hand-authoring that one cross-service check and hoping someone remembers to update it, connecting a codebase to Autonoma turns system-level coverage into something derived from the code instead of remembered by whoever wrote it first.
Frequently Asked Questions
Integration testing checks that two or more modules honor the contract they agreed on when wired together. System testing checks the fully assembled product, every module included, against its own requirements, treating the whole thing as one black box rather than checking any single pair. A build can pass every integration test between its services and still fail a system test, because the violation only shows up once every piece is assembled and run together.
A checkout flow assembled from a cart, pricing, inventory and payment service, driven end to end as a real customer would use it: add items, apply a promo code, confirm the order, then verify that the amount actually charged matches the total shown before checkout. That single assertion can catch a requirement violation that three passing integration tests, each checking one service pair in isolation, never would.
It can be either. Historically, system testing was run manually by a QA team stepping through the assembled product the way a user would, because writing and maintaining a script that walks the whole flow, across every service boundary, was expensive to do by hand. It is automatable the same way any other testing level is, provided something can generate and maintain the checks as the assembled flow changes.
No. System testing verifies the assembled product against its written requirements. Acceptance testing, whether that's a UAT pass or a beta program, verifies that real users or the people who commissioned the product accept it as fit for their actual purpose. A build can pass every system test and still fail acceptance if it technically meets its spec but isn't what the business or the user actually needed.
By testing the assembled product as a whole rather than any one service pair. Integration tests validate a contract between two components in isolation. A bug that emerges only from how three or more correctly-behaving components compose together, like a discount applied twice across a checkout flow, is invisible to any of them individually and only shows up once something drives the literal, assembled user path and checks the requirement that spans all of it.
Yes, in the sense used here: Autonoma drives the fully assembled, deployed product as a black box and checks it against flows derived from your codebase, which is exactly what a system test does. It is not a replacement for the load-testing tool, accessibility scanner, or unit runner that a complete system-testing effort still needs running alongside it. Those stay separate tools answering separate questions.
