Risk-based test selection is the per-commit decision of which tests actually run against a given deploy, chosen from signals like the files changed, how far that change reaches through the dependency graph, and past co-failure history, instead of running the entire suite every time. It differs from prioritization, which orders tests once a set is chosen, and from deciding whether a test should exist at all, which is a separate scoring question.
Your full suite does not fit in the window between a merge and a deploy. It probably stopped fitting a while ago, and the workaround was moving it to a nightly job instead of a per-commit gate in the CI/CD pipeline, which quietly pushed the safety net two steps back from where anyone would notice a regression land.
This is for the QA lead, QA manager, or engineering manager who owns quality without a dedicated QA function, somewhere between Series A and Series C, with a real suite and a real release cadence. It is not for a team with no tests yet; that's an earlier conversation with a different document. It is not for a team trying to figure out whether an AI-generated test is actually testing anything; that's a different one too. The trigger here is more specific: someone above you asked for a document that says what gets tested on a given deploy, and "we run everything" stopped being a credible answer once the suite outgrew what a commit-triggered pipeline can absorb.
Selection, prioritization, and existence are three different questions
Three questions get flattened into one in most strategy documents, and the flattening is where teams lose the thread.
Whether a test should exist at all is a scoring question, decided by weighing likelihood of failure against blast radius and revisited periodically rather than per commit. Risk-based testing covers the two-axis model and the worked scoring table; that's the reference for building the suite in the first place, and this article doesn't restate it.
What order the chosen tests run in, when the full set won't fit inside a CI window, is a sequencing question. Test case prioritization covers coverage-based, history-based, cost-aware, and similarity-based ordering; that's a separate lever from the one this article pulls.
What actually runs on this deploy, out of tests that already exist, is the question here. It gets decided fresh on every commit, from signals about the specific change, not from a static score assigned to the test months ago.
The five inputs a selector can actually read
A selector that only knows a commit happened can't make this decision on its own. It needs five signals, and each one is available from infrastructure most CI/CD pipelines already run.
Files changed in the diff
The most basic input is also the only one that's ground truth: what actually moved. A selector reads it straight off git diff --name-only against the merge base, before anything else runs. Everything downstream of this input is inference; this one is a fact about the commit.
Dependency-graph reach
Knowing which files changed says nothing about what depends on them. A selector needs the reach: how far a change propagates through imports, calls, or build targets. Most toolchains already build this graph for other reasons: an import graph in a JavaScript bundler, a call graph in a compiled language, or a reverse-dependency query in a build system like Bazel. The selector doesn't build a new graph; it queries the one that already exists to compute a build.
Historical co-failure
Some tests fail alongside changes in a given area more often than the dependency graph alone would predict, usually because of a shared fixture, a flaky third-party call, or a coupling the code doesn't formally declare. That signal comes from joining CI result history to the commits that triggered each run: which tests failed, and what changed in the commit right before the failure. It's the input most teams already have the data for and rarely query, because it lives split across a CI provider's API and wherever commit metadata is stored.
This is also the input that improves with age. A brand-new pipeline has no co-failure history worth reading, so early on the dependency graph is doing most of the work. Six months and a few thousand runs later, the co-failure signal starts catching couplings the graph was never going to see, which is a reason to revisit the selection logic on a schedule rather than treat it as something you configure once and leave alone.
Feature-flag state
A change gated behind a flag that's off in the target environment doesn't need its suite run there. That comes from the flag provider's evaluation state for the environment being deployed to, not from the code itself. Two deploys of the same commit, one with a flag on and one off, are legitimately different selection decisions.
Time since the last full run
Every input above narrows the set, and narrowing compounds. Left alone, it compounds into blind spots that never get corrected, because nothing in the selection logic ever asks to look at everything again. The fifth input is a decay term: track how long it's been since the full suite ran, and force a full run once that number crosses a threshold, independent of what diff-based selection would have chosen. It's the one input that isn't about the current commit at all; it's about the health of the process.
Two of the three changed files reach tests through import edges. The config change reaches nothing, so the selector picks nothing for it.
The tiering that follows
Those five inputs feed a small number of tiers, not a continuous score. In practice, three is enough. This is the shape most regression testing in CI/CD converges on once a suite outgrows its window.
| Tier | Trigger | Scope | Target duration | What it does not catch |
|---|---|---|---|---|
| Smoke | Every commit | Critical path only | 2-5 min | Anything outside the core flow |
| Targeted regression | Merge to main | Reached surface from diff | 10-20 min | Unreached blind spots (config, data) |
| Full suite | Nightly or pre-release | Entire suite | Hours, off critical path | Anything broken since last full run |
Target durations here are engineering targets we'd defend in a review, not benchmarks pulled from a vendor page. Your numbers will differ with suite size and CI hardware; the shape of three tiers, not the exact minutes, is the part worth keeping.
Each tier fires at a different point in the pipeline, and only the first two sit on the critical path between a commit and a deploy.
How Autonoma decides what to re-run
Everything above assumes your team owns the machinery that reads diffs, walks dependency graphs, and joins CI history to commits. Most teams don't, not because it's hard to conceive of, but because building and maintaining that machinery becomes its own project, on top of the testing it was supposed to make faster.
We built Autonoma as the end-to-end testing layer that generates and runs tests directly from your codebase, and the same diff-reading the five inputs above depend on is structural to how it maintains that suite. The Diffs Agent runs on every PR, reads what changed, and updates the test suite against it: adding tests for new surface, deprecating tests for surface that's gone, and re-running the tests a given diff actually implicates rather than the entire suite by default. That's the selection decision, applied specifically to the E2E layer, without a separate selector to build and tune.
It's a partial answer to the five inputs, not a replacement for all of them. The Diffs Agent's diff-to-test mapping covers the E2E slice of files-changed and dependency-reach: what changed, and which of its own tests that change touches. It doesn't own unit-level test selection inside your language toolchain, your build system's reverse-dependency queries, or your feature-flag provider's evaluation state. Those stay wherever they already live, and a real risk-based selection strategy still reads all five inputs across every layer, not just the one Autonoma runs on.
The trade you are making
Selection is a deliberate trade of safety for speed, and it's worth stating as a trade rather than a solved problem, because it has a real failure mode: a change whose blast radius the dependency graph can't see.
The classic case isn't in the graph at all. A config value, a database migration, an infrastructure change, or a third-party or CDN-level shift doesn't show up as an edge between files, so a selector that only reads import graphs will confidently select nothing for it. The suite that would have caught it never runs, and the pipeline stays green the whole way through.
This is why the decay term, the fifth input, matters more than it looks like it does on a slide. It isn't a nice-to-have; it's the mitigation for the one failure mode selection can't design around. A full run on a fixed cadence, independent of what diff-based selection decided, is the backstop for exactly the changes this whole system is structurally blind to.
Data and infrastructure changes deserve a specific callout because they fail differently from code. A bad migration or a misconfigured CDN rule doesn't throw at deploy time the way a broken function call does; it surfaces minutes or hours later, once real traffic hits the changed path, which is exactly the delay a smoke suite running in a five-minute window will not catch. Treat those categories as an automatic trigger for a broader run regardless of what the dependency graph says, rather than trusting the graph to flag them on its own.
It's also why this belongs written into the strategy document, not buried in a CI YAML file that only the person who wrote it ever opens again. A selection strategy is a risk decision made on behalf of the whole team, and a decision that consequential should be visible to whoever signs off on the release process, reviewable when the trade stops being worth it, and owned by someone other than whoever configured the pipeline two years ago. A strategy document is really an allocation plan for a scarce resource, and what's scarce here isn't the time to write the tests anymore; it's the attention someone has to spend deciding, deploy by deploy, which of them still deserve to run. Writing that allocation down is what turns it back into a decision instead of a default. The same logic carries into how you read the test automation metrics that predict release quality that come out the other end: a rising failure signal against a green pipeline is often selection quietly missing something, not a coincidence.
None of this replaces judgment. The five inputs narrow the field; someone still decides the tiers, the thresholds, and how often "nightly" actually means nightly under load. What changes when you write it down is that the trade becomes a decision your team made on purpose, instead of a default nobody remembers choosing, and one an E2E testing strategy for AI teams has to name explicitly rather than inherit from a template written before generation was cheap.
If part of that trade is the end-to-end layer specifically, Autonoma is the piece we'd point you at: it already reads the diff to decide what to maintain and re-run on that layer, so the selection logic for your generated E2E suite doesn't need a separate build. The other inputs, across the rest of your suite, are still yours to wire up, and they belong in the same document as this one, not scattered across CI configuration and regression suite ownership that nobody reviews together. A pipeline that stays green because it stopped looking isn't the zero-bug outcome anyone was aiming for.
Frequently Asked Questions
Risk-based test selection is the practice of choosing which tests actually run against a specific commit or deploy, based on signals like which files changed, how far that change reaches through the dependency graph, and which tests have historically failed alongside similar changes, rather than running the full suite every time.
By reading five inputs at deploy time: the files changed in the diff, the dependency-graph reach of those files, historical co-failure data from CI results, the feature-flag state for the target environment, and how long it has been since the full suite last ran. Those inputs feed a small number of tiers, typically smoke on every commit, targeted regression testing on the reached surface, and a full run on a fixed cadence.
It is safe in proportion to how honestly you account for its blind spot: changes a dependency graph cannot see, like config values, database migrations, infrastructure changes, and third-party or CDN-level shifts. Selection is a deliberate trade of safety for speed, and the mitigation is a full run on a cadence independent of what diff-based selection decided, not a claim that selection alone is complete.
Selection decides which tests run at all, out of the tests that already exist, based on signals about the current commit. Prioritization decides the order the selected tests run in, when the full set will not fit inside a CI window. They solve different constraints: selection reduces scope, and prioritization sequences whatever scope remains.
Often enough that the gap between full runs is shorter than how long a selection-induced blind spot could go unnoticed, commonly nightly or on a pre-release cadence. The exact interval matters less than having one enforced independent of diff-based selection, since that decay term is what catches the changes selection structurally cannot see.
For the end-to-end tests it generates and maintains, yes: the Diffs Agent reads each PR's diff and re-runs, adds, or deprecates the tests that diff actually implicates, which is a selection decision applied to that layer. It does not replace a full five-input selection strategy across your whole suite. It does not own unit-level selection inside your language toolchain, your build system's dependency queries, or your feature-flag provider's state. Those still belong to whoever owns those layers.
