OAT testing (operational acceptance testing) is the acceptance gate that checks whether a product can actually be operated, not whether it behaves correctly: can it be deployed, backed up and restored, failed over, monitored, patched, and rolled back without someone improvising at 2am. It sits next to UAT as one of the named acceptance types, but where UAT checks business functionality against real users, OAT checks operability against the team that has to keep the thing running.
A feature ships. It passes every functional check a UAT session could ask for: the workflow works, the numbers reconcile, the stakeholders sign off. Three weeks later a database node fails over during a Tuesday deploy, and it turns out nobody had actually run a restore from that environment's backups since the project started. The runbook exists. It references an endpoint that moved four sprints ago. It's the same gap Google's SRE team addresses under data integrity practices: a backup that's never been restored has never actually been verified.
Nothing in the UAT sign-off would have caught that, because UAT was never asking that question. That's the gap OAT testing exists to close, and this is written for the engineer, SDET, or release owner who has to decide whether a build is operationally safe to ship, or who was just handed the job of naming and running that gate. It is not written for the QA lead building a suite-allocation strategy, the person asking whether an AI-generated test is asserting anything real, or someone choosing inputs for a single test case. Those are real questions, just not this one.
What OAT Testing Actually Checks
Most definitions stop at "OAT checks that the system meets operational requirements" and leave the reader to guess what an operational requirement is. In practice it's a short, checkable list: can the system be installed and configured the way the operations team actually installs things, can it be backed up and restored, can it fail over, can it be monitored so a real incident produces a real alert, can it be patched and rolled back, and can someone who isn't the original author operate it from the runbook alone.
That list sits inside a wider map of acceptance types (business acceptance, contract acceptance, regulatory acceptance, and OAT among them), which the acceptance testing hub lays out in full. The distinction that matters here is narrower: functional acceptance testing, most visibly user acceptance testing, asks whether the product does the right thing for the person using it. OAT asks whether the product can be kept alive by the people who have to run it after launch. A build can pass UAT cleanly and still be operationally unsafe to ship, and the reverse also happens, an operationally solid build that fails UAT because a workflow doesn't match how the business actually works.
Same word, two audiences. UAT answers to the user. OAT answers to whoever is on call.
Pull one thread further and a second distinction shows up underneath both: verification asks whether the build matches its specification, validation asks whether it matches what the user actually needed. Most of an operability check is verification. "Does a restore from this backup produce a running system" has a defined right answer in the spec for the backup process, which is exactly why it's derivable and checkable by a machine. A UAT session, by contrast, leans validation: no artifact in the repository says whether the workflow matches how a real business user thinks about their job, so a person still has to judge it. That's the deeper reason OAT automates further than UAT does. It isn't that operability is simpler; it's that operability is specified, and specified things are cheap to generate checks for.
The 12-Point Operational Acceptance Testing Checklist
Every OAT glossary defines the gate and then stops before the part that's actually useful: the filled checklist. Here's one built against a real release process: twelve operability checks, split into two halves of six. The first covers whether the service survives what happens to it. The second covers whether the people running it can see what's happening and control it. This is the artifact a release owner actually needs at the gate, not a suite-allocation strategy for a QA team and not the boundary values inside one test case, both of which are different questions answered elsewhere. The idea of gating a launch behind a structured checklist isn't new either: Google's SRE book documents a similar launch checklist practice for reviewing services before they go live.
Survivability Checks: Backup, Failover, Rollback, Capacity
| Operability check | Evidence it passed | Owner | How it's verified |
|---|---|---|---|
| Backup taken, restore verified | Restore ran, correct data came back | SRE / infra | Restore drill on scratch environment |
| Failover tested | Traffic moved to secondary and stayed served | SRE / platform | Game-day failover exercise (manual) |
| Rollback works | Previous version restored, no manual cleanup | Release engineering | Automated rollback on live preview |
| Deploy is repeatable | Same artifact redeploys to identical result | Platform / release | Automated re-deploy comparison |
| Patch and upgrade path works | Patch applied without unplanned outage | Platform / security | Patch applied to staging, re-verified |
| Capacity headroom exists | Load sits well below the failure point | SRE / capacity planning | Load test vs measured peak traffic |
Control Checks: Runbook, Alerting, Logs, Access, Migration
| Operability check | Evidence it passed | Owner | How it's verified |
|---|---|---|---|
| Runbook exists and is current | Current, and followable by someone new | On-call / owning team | Dated review, fresh execution test |
| Alerting actually fires | Injected failure triggers a real page | On-call / SRE | Fault-injection test, not a checkbox |
| Logs are accessible | Engineer finds the incident log line fast | SRE / observability | Timed log-retrieval drill |
| Config and secrets load correctly | Required keys present at boot, none hardcoded | Platform | Automated boot-time secret check |
| Data migration is reversible | Migration rolls back, no data loss | Release engineering / DBA | Rollback run against copied prod data |
| Access control and audit trail | Access matches policy, actions logged | Security / platform | Access review, audit-log spot check |
Read the third column across both tables and a pattern shows up: half of it says "automated," half says "manual," and neither half is being modest about which is which. A failover game day is a genuine drill involving people and a war room, and pretending a script can substitute for it is how a team discovers its failover plan doesn't work during the actual incident. A boot-time secret check is genuinely a machine's job, and pretending it needs a human to click through it every release is how that check silently stops running.
A backup that was taken but never restored has not been verified. It has only been stored.
The gate sits between functional acceptance and production. What it checks splits cleanly into what a machine can now verify and what a person still has to rehearse.
That split is the actual shift underway. When a human ran every row of this checklist by hand, once, ahead of a launch, the whole thing was a hand-curated document that went stale the moment a route or a config key changed. When the derivable rows are generated from the current codebase and re-checked on every change instead of once before launch, OAT stops being a document you maintain and becomes a signal you regenerate. The scarce work moves from writing the checks to reviewing what they surface, which is a very different job for the release owner than the one the classic OAT testing glossaries describe.
How Autonoma Tests the Operable Behaviours
Look back at that checklist and a line runs through it: several rows are asking whether a running application, poked the right way, produces the right observable response. Does the auth path still exist. Does a broken dependency surface as a real page instead of a silent timeout. Does a config value actually load before the process claims to be ready. Those rows aren't checking the infrastructure underneath the app; they're checking behaviour the app itself exposes, and they're exactly the rows a human had to keep re-picking as routes moved and config keys got renamed.
That's the half we built Autonoma to own. Connect a repository and our agents read the routes, forms, and flows that exist in the codebase this week, then run those checks against a live, deployed preview instead of a saved script, so a rollback or a config check keeps working after a route gets renamed instead of quietly failing the next release. The Diffs Agent re-reads every pull request and updates the covered behaviours accordingly, so the behavioural rows on an operability checklist stay aligned with what the app actually does, instead of testing a login flow that moved two sprints ago. None of that involves a benchmark or a speed claim; it's an architecture decision about where the checks come from, not a number we're asking you to trust.
What Autonoma Doesn't Test: The Ops-Owned Drills
Being specific about the line matters more here than almost anywhere else in this cluster, because an operability checklist is exactly where a vendor is tempted to claim the whole table. Autonoma verifies the behaviours a running application exposes. It does not run your infrastructure drills.
A full disaster-recovery exercise, the kind where a region goes dark on purpose and the whole team executes the actual recovery plan, is a human rehearsal, and the value is in the humans discovering where their own plan breaks under pressure. A regional failover game day is the same category: the point is testing the team's response, not just the system's. On-call escalation rehearsal, confirming the right person actually gets paged and knows what to do, is a process exercise, not a behavioural check. Backup media integrity at the storage layer, verifying the physical or cloud-storage durability of a backup blob, sits below the application layer entirely, and belongs with your storage vendor's own tooling or a dedicated backup-verification product. Capacity and load headroom is a load-testing discipline with its own tools built for sustained traffic simulation, not for checking whether one flow behaves correctly.
One vendor covering every row on this list would be a red flag, not a feature.
None of that is a gap in Autonoma. It's a boundary. Pair the behavioural checks Autonoma runs on every preview with the drills your team already owns, the same way you'd pair a smoke suite with a manual beta program rather than expect one to replace the other.
Where OAT Testing Sits Next to the Other Gates
Operational acceptance testing isn't the only acceptance type, and it isn't the last gate before production either. The acceptance testing hub maps how OAT relates to business, contract, and regulatory acceptance, and to functional gates like user acceptance testing, so start there if you're trying to name which acceptance type you're actually being asked to run. If the build already cleared build verification testing and a functional acceptance round, OAT is typically the last checkpoint before the rows on this checklist get folded into a broader release readiness checklist that a release owner signs off against. Some of what OAT checks pre-production keeps getting re-checked after launch too, which is the territory testing in production covers. And if the gate is being wired into a pipeline so a failed row actually blocks a promote, that wiring is the CI/CD testing guide and, for teams on Vercel specifically, the deployment checks guide.
None of those pages replace the checklist above. They tell you what happens to a build before it reaches this gate, and what happens to the rows on it after a release owner signs off.
If you're standing at a release gate right now trying to decide whether a build is operationally safe, the two tables above are the actual job, whatever tool eventually runs them. The rows that ask whether the running application behaves correctly under a specific poke, an auth check, a config load, a health response, are the ones worth automating first, because they're the ones that silently drift every time the codebase changes underneath them. That's the layer Autonoma was built to keep current without anyone having to remember to update a checklist by hand.
Frequently Asked Questions
OAT testing, short for operational acceptance testing, is the acceptance gate that checks whether a product can be operated once it's live: deployed, backed up and restored, failed over, monitored with alerts that actually fire, patched, and rolled back. It's distinct from functional acceptance testing (like UAT), which checks whether the product does the right thing for a user rather than whether it can be kept running by the team responsible for it.
UAT (user acceptance testing) checks business functionality against real users: does the workflow do what the business needs. OAT checks operability against the operations team: can the system be deployed, backed up, monitored, and recovered. A build can pass one and fail the other. They're both acceptance types, but they answer to different audiences and check different things entirely.
A usable checklist covers whether the service survives what happens to it (backup taken and restore verified, failover tested, rollback works, repeatable deploys, a working patch path, and capacity headroom) and whether the people running it can see and control it (a current runbook, alerting that actually fires under a real fault, accessible logs, correctly loaded config and secrets, a reversible data migration, and an audit trail matching access policy). Each row needs a pass condition that's actual evidence, not a checkbox.
Partially, and increasingly so. Rows that check specified, derivable behaviour, like whether a rollback restores service or whether an alert fires when a fault is injected, are the ones a machine can now generate and re-run continuously. Rows that are genuinely human rehearsals, a full disaster-recovery exercise, a regional failover game day, on-call escalation practice, stay manual on purpose, because the value is in the team practicing, not just the system responding.
Autonoma covers the application-behaviour rows on an OAT checklist, the part that is easiest to let go stale. Its agents read your codebase and verify behaviours a running application exposes, whether a config value loads correctly, whether an auth path survives a rollback, whether a broken dependency surfaces as a real error instead of a silent timeout, by running checks against a live preview on every pull request. The infrastructure drills, disaster-recovery game days, failover exercises, load testing, stay with your ops and SRE tooling, where the value is in people rehearsing their own plan. Point Autonoma at the behavioural half and it keeps those rows verified continuously, so your ops team's time goes to the drills only humans can run.
It's typically split. Release engineering and platform teams own deploy repeatability, rollback, and patch paths. SRE or infrastructure owns backup, restore, and failover. On-call and the owning team keep the runbook current and confirm alerting fires. Security and platform own access control and audit trails. OAT rarely has a single owner precisely because it spans everything the operations side of a launch is responsible for, unlike UAT, which is typically owned by the business stakeholders.
A restore drill is a good concrete example. A team takes a backup from the release candidate's environment, restores it into a scratch environment, and confirms the restored system boots and returns the correct data. The pass condition is the restore having actually run and produced a working system, not a backup job reporting green. The common failure this catches: nightly backups that have run successfully for a year and never once been restored, which means nobody actually knows if they work.
No. Non-functional testing is a broad category covering performance, security, usability, and reliability characteristics of the product itself. OAT is an acceptance gate with a specific audience: it asks whether the operations team can actually run the thing. They overlap in places, capacity headroom shows up in both, but OAT also covers process artifacts like runbooks and on-call alerting that no non-functional suite touches, and it excludes plenty of non-functional testing that happens long before any acceptance gate is reached.
