How to Test AI Agents That Take Actions (Tool Calls)

Testing tool calling agents means verifying the trajectory an agent produces when it acts: the ordered list of tool calls it made, with what arguments, captured and inspected after the fact. This is trajectory evaluation, a white-box check on tool selection, call order, and argument accuracy. An agent can pass all three and still be wrong, because none of them confirm the tool's effect on the running application was correct.

Search "trajectory evaluation" right now and you'll get the concept explained cleanly, in a vendor glossary, a framework doc, a conference talk transcript. What you won't get, in almost every case, is a test file. The term is well-covered. The assertions are not, and that gap is exactly where a team with a tool-calling agent in production gets stuck: they know what to check, they don't have anything to paste into their CI config.

This is that code. Every check below is real Python against a small harness in the companion repo, no SDK, no platform sign-up. The throughline: an agent can select the correct tool and still be wrong, because the argument it passed was wrong, and the two failures look identical from far away and completely different once you write the assertion.

What Is Trajectory Evaluation?

A trajectory is just the list your agent produces as it works: which tools it called, in what order, with what arguments, and what came back. Trajectory evaluation is the practice of capturing that list and asserting against it after the run, instead of eyeballing a transcript or trusting that "it seemed to work."

That distinction matters because a tool-calling agent has a failure surface a plain function call doesn't. A function returns the right value or it doesn't. An agent decides which function to call, in what order relative to others, and what to pass it: three decisions that fail independently, usually in this order of frequency: wrong arguments most often, wrong order less often, wrong tool rarest once a system has real usage behind it.

Trajectory evaluation covers tool selection, sequence, and arguments, the first three checkpoints. It stops at the boundary of the call. It never sees whether the tool's effect on the running application was actually correct.

If your agent talks to tools over MCP rather than in-process function calls, the protocol and handshake layer underneath these checkpoints is its own testing surface, covered in our guide to testing an MCP server. LangGraph formalizes a version of this same idea as a first-class concept in its evaluation tooling; if you're on that framework specifically, our LangGraph testing guide covers the framework-specific plumbing. Everything below is intentionally framework-neutral: an OpenAI-style tool-calling shape, a generic agent loop, no SDK required to follow along.

The Harness: A Framework-Neutral Agent Loop You Can Actually Test

Every assertion in this guide runs against the same small harness: an Agent class that loops until it decides it's done, calling real tool functions and recording each call, its arguments, its result, and any error, into a trajectory. The only seam is llm_decide, a callable that looks at the request and the trajectory so far and returns the next action. Swap it for a real model call in production. In tests, it's a scripted fake, which is what makes the trajectory fully deterministic and inspectable instead of a black box you're hoping behaved.

"""
Minimal, framework-neutral agent loop for testing tool-calling behavior.

This harness mirrors the shape of any OpenAI-style tool-calling agent: the
LLM sees the user request and the trajectory so far, decides on a tool call
(or decides it's done), the tool executes against real application state,
and the result feeds back into the next decision.

Swap `llm_decide` for a real model call in production. For tests, it's a
scripted or fake callable, which is the whole point: the trajectory becomes
fully deterministic and inspectable.
"""
from dataclasses import dataclass
from typing import Any, Callable, Optional


@dataclass
class ToolCall:
    """One step in the agent's trajectory: what was called, with what
    arguments, and what came back (a result, or an error)."""
    tool_name: str
    arguments: dict
    result: Any = None
    error: Optional[str] = None


class ToolError(Exception):
    """Raised by a tool when it fails. Caught by the agent loop, not the caller."""


class Agent:
    """
    Drives a tool-calling loop against a set of real tool functions.

    `llm_decide(request, trajectory, tool_names) -> dict | None` is the only
    seam. It returns {"tool": name, "arguments": {...}} to call a tool, or
    None to signal the agent is done.
    """

    def __init__(self, tools: dict[str, Callable[..., Any]], llm_decide: Callable, max_steps: int = 6):
        self.tools = tools
        self.llm_decide = llm_decide
        self.max_steps = max_steps
        self.trajectory: list[ToolCall] = []

    def run(self, user_request: str) -> list[ToolCall]:
        self.trajectory = []

        for _ in range(self.max_steps):
            decision = self.llm_decide(user_request, self.trajectory, list(self.tools.keys()))
            if decision is None:
                break

            tool_name = decision["tool"]
            arguments = decision.get("arguments", {})
            call = ToolCall(tool_name=tool_name, arguments=arguments)

            if tool_name not in self.tools:
                call.error = f"unknown tool: {tool_name}"
                self.trajectory.append(call)
                continue

            try:
                call.result = self.tools[tool_name](**arguments)
            except ToolError as exc:
                call.error = str(exc)

            self.trajectory.append(call)

        return self.trajectory

The fixtures underneath it are just as small: an in-memory AppState standing in for your real application (here, a toy booking system with a bookings dict), three tool functions (search_flights, book_flight, get_weather) that read and write that state, and a ScriptedLLM fake that plays back a fixed list of decisions so every test below is reproducible without a live model call:

"""
Shared fixtures for every test file in this repo: a tiny in-memory
application (AppState), the three tools the agent can call, an agent
factory, and a ScriptedLLM fake that plays back a fixed list of decisions
so trajectories are fully deterministic in tests.
"""
import pytest

from agent import Agent, ToolError


class AppState:
    """The 'application' the tools act on. Tests assert against this to
    check outcomes, not just that a tool call fired."""

    def __init__(self):
        self.bookings: dict[str, dict] = {}
        self._next_id = 1
        self.flight_error_countdown = 0  # used by failure-injection tests

    def next_booking_id(self) -> str:
        booking_id = f"BK-{self._next_id:04d}"
        self._next_id += 1
        return booking_id


@pytest.fixture
def app_state():
    return AppState()


@pytest.fixture
def tools(app_state):
    def search_flights(origin: str, destination: str, date: str):
        return [
            {"flight_id": "FL-100", "origin": origin, "destination": destination, "date": date, "price": 220},
            {"flight_id": "FL-200", "origin": origin, "destination": destination, "date": date, "price": 340},
        ]

    def book_flight(flight_id: str, date: str, passenger_name: str):
        if app_state.flight_error_countdown > 0:
            app_state.flight_error_countdown -= 1
            raise ToolError("booking-service-timeout")
        booking_id = app_state.next_booking_id()
        app_state.bookings[booking_id] = {
            "flight_id": flight_id,
            "date": date,
            "passenger_name": passenger_name,
        }
        return {"booking_id": booking_id, "status": "confirmed"}

    def get_weather(city: str):
        return {"city": city, "forecast": "clear", "high_f": 72}

    return {
        "search_flights": search_flights,
        "book_flight": book_flight,
        "get_weather": get_weather,
    }


@pytest.fixture
def make_agent(tools):
    def _make(llm_decide, max_steps: int = 6):
        return Agent(tools=tools, llm_decide=llm_decide, max_steps=max_steps)
    return _make


class ScriptedLLM:
    """A fake 'model' that plays back a fixed list of decisions, one per
    call, standing in for a real tool-calling LLM so trajectories are
    deterministic and inspectable in tests."""

    def __init__(self, script: list):
        self.script = list(script)
        self.calls = 0

    def __call__(self, request, trajectory, tool_names):
        if self.calls >= len(self.script):
            return None
        decision = self.script[self.calls]
        self.calls += 1
        return decision

book_flight writes to AppState.bookings, real application state, which is what lets the mocking-vs-live section later demonstrate the difference between asserting on a call and asserting on an effect.

Assertion One: Did the Agent Call the Right Tool

The cheapest and most common trajectory assertion, and the one nearly every existing write-up stops at: given a booking request, does book_flight show up in the trajectory at all, and does an irrelevant tool like get_weather stay out of it.

"""
Assertion type 1: did the agent call the RIGHT tool.

This is the cheapest, most common trajectory assertion, and the one every
tutorial stops at. It only tells you tool SELECTION was correct; it says
nothing about arguments or outcome. Keep it, but don't stop here, see
test_argument_accuracy.py for why.
"""
from conftest import ScriptedLLM


def test_agent_calls_book_flight_for_a_booking_request(make_agent):
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book me on flight FL-100 for August 1st, name A. Rivera")

    tool_names = [step.tool_name for step in trajectory]
    assert "book_flight" in tool_names, f"expected book_flight in trajectory, got {tool_names}"


def test_agent_does_not_call_weather_for_a_booking_request(make_agent):
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book me on flight FL-100 for August 1st, name A. Rivera")

    tool_names = {step.tool_name for step in trajectory}
    assert "get_weather" not in tool_names

This test is worth having. It catches an agent that reaches for entirely the wrong capability, which does happen, especially as the tool list grows and tool descriptions start overlapping. It's also the least informative check in this guide, because it says nothing about whether the call that did happen was actually useful. That's the next two sections.

Assertion Two: Did It Call Them in the Right Order

Tool selection can be perfect and the trajectory can still be broken if the sequence is wrong. Search-then-book is an obvious dependency (you can't book a flight ID you haven't looked up), but sequence assertions matter even without a strict data dependency: a weather check the user asked for before booking, if it runs after the booking already happened, is not the trajectory anyone asked for, even though every individual tool call, in isolation, looks fine.

"""
Assertion type 2: did the agent call tools in the RIGHT ORDER.

Selecting the right tools isn't enough when a later call depends on an
earlier one's output. Booking before searching, or searching after the
booking already happened, both pass a "right tool" check while producing
a broken trajectory.
"""
from conftest import ScriptedLLM


def test_agent_searches_before_booking(make_agent):
    llm = ScriptedLLM([
        {"tool": "search_flights", "arguments": {"origin": "SFO", "destination": "JFK", "date": "2026-08-01"}},
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Find me a flight SFO to JFK on Aug 1 and book the cheapest one for A. Rivera")

    tool_sequence = [step.tool_name for step in trajectory]
    assert tool_sequence == ["search_flights", "book_flight"], (
        f"expected search before book, got {tool_sequence}"
    )


def test_full_trip_planning_keeps_weather_check_before_booking(make_agent):
    """A three-tool trajectory. Order still matters even when none of the
    later calls strictly depend on the earlier ones' output, because a
    weather check the user asked for that runs AFTER the booking already
    happened is not the trajectory they asked for."""
    llm = ScriptedLLM([
        {"tool": "search_flights", "arguments": {"origin": "SFO", "destination": "JFK", "date": "2026-08-01"}},
        {"tool": "get_weather", "arguments": {"city": "New York"}},
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run(
        "Check the weather in New York, find a flight SFO to JFK on Aug 1, "
        "and if it looks clear book the cheapest one for A. Rivera"
    )

    tool_sequence = [step.tool_name for step in trajectory]
    assert tool_sequence == ["search_flights", "get_weather", "book_flight"], (
        f"expected search, then weather, then book, got {tool_sequence}"
    )

The second test in that file uses a three-tool trajectory on purpose. Two-tool sequencing tests pass trivially once you've written one; three tools is where teams actually get bitten, usually by an agent that reorders steps under a slightly rephrased prompt.

Assertion Three: Did It Call Them With the Right Arguments

This is the deepest section in this guide for a reason: argument accuracy is the highest-value trajectory assertion and the one almost every public write-up skips entirely. Tool selection and order both check that the right shape of thing happened. Argument accuracy checks whether it happened correctly, and it's where "the agent worked" and "the agent did what the user asked" stop being the same claim.

"""
Assertion type 3: did the agent call the tool with the RIGHT ARGUMENTS.

This is the highest-value, most-skipped check. An agent can select the
correct tool, in the correct order, and still produce the wrong outcome if
the arguments are wrong: wrong date, wrong id, wrong recipient. "It called
book_flight()" and "it booked the flight the user actually asked for" are
different claims. Only this layer of assertion tells them apart.
"""
from conftest import ScriptedLLM


def _find_call(trajectory, tool_name):
    for step in trajectory:
        if step.tool_name == tool_name:
            return step
    return None


def test_book_flight_called_with_requested_date(make_agent):
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for August 1st 2026, passenger A. Rivera")
    call = _find_call(trajectory, "book_flight")

    assert call is not None
    assert call.arguments["date"] == "2026-08-01", (
        f"agent picked the right tool but the wrong date: {call.arguments['date']}"
    )


def test_book_flight_called_with_requested_passenger_name(make_agent):
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for August 1st 2026, passenger A. Rivera")
    call = _find_call(trajectory, "book_flight")

    assert call.arguments["passenger_name"] == "A. Rivera"


def test_right_tool_wrong_argument_is_still_a_real_failure(make_agent):
    """
    The through-line of this guide: right tool, wrong argument, wrong
    outcome. A tool-selection-only test would pass on this trajectory. This
    test proves the argument-accuracy check catches what that one misses.
    """
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-02", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for August 1st 2026, passenger A. Rivera")
    call = _find_call(trajectory, "book_flight")

    tool_selection_passed = call is not None and call.tool_name == "book_flight"
    argument_accuracy_passed = call.arguments["date"] == "2026-08-01"

    assert tool_selection_passed is True
    assert argument_accuracy_passed is False, (
        "tool selection passed, but the date argument is wrong: this is the "
        "right-tool-wrong-outcome failure mode a tool-selection-only test misses"
    )

The third test in that file is deliberately the point of this whole guide: it books a flight one day off from what the user asked for, confirms tool selection passed, then confirms the argument-accuracy check correctly fails. If you only ship the first assertion type from this guide, ship this one instead. "It called book_flight()" is not a finding. "It called book_flight() with the date the user actually asked for" is.

Right Tool, Right Order, Right Arguments: Still Not the Same as Right Outcome

Run every assertion above and get green across all of them, and here's what you actually know: the agent picked the correct tool, in the correct sequence, with arguments that match what the user asked for. Here's what you still don't know: whether that tool call changed anything correctly in the running application the user is looking at. The trajectory tests in this guide are white-box, they inspect the call. None of them touch the app itself.

That gap is easy to miss because it's invisible from inside the trajectory. A book_flight() call with the exactly right arguments can still fail if the booking service silently drops the write, if a downstream queue never processes it, or if the UI the user actually checks afterward never reflects the new state. Trajectory assertions have no way to see any of that, because by design they stop at the boundary of the call.

The agent picked the right tool. A tool-selection-only test passes here. The argument is still wrong, and so is the outcome in the running application, which only an argument-accuracy assertion catches.

How Autonoma Verifies What the Tool Call Actually Did

Every test in this guide answers a version of the same question: did the agent fire the right call, in the right order, with the right arguments. None of them answer whether the call's effect showed up correctly where the user would actually look for it, a booking in the itinerary, a record in a dashboard, a state change reflected back in the UI. That's a different layer of testing, and it's the one Autonoma covers.

We built Autonoma to run behavioral, end-to-end tests directly against a running application rather than against a captured trajectory. Our Planner reads the codebase, the routes, the components, the flows a tool call is supposed to trigger, and plans test cases (including the database state each scenario needs) around what should actually happen after an action fires. The Execution Agent then drives the real UI in a live environment to confirm it: where a trajectory test in this guide asserts book_flight() was called with {'{'}date: '2026-08-01'{'}'}, the behavioral test asserts the booking actually appears in the itinerary screen with that date. The Diffs Agent keeps that coverage current as the codebase changes, so it doesn't quietly rot the next time a tool's downstream behavior shifts.

Mapped against this guide directly: the six assertion types above tell you the call was well-formed. Autonoma tells you the thing the call was supposed to do actually happened, correctly, where a user would see it. A tool-calling agent worth shipping needs both. Trajectory tests alone can't tell you the second half, and outcome tests alone would tell you something broke without narrowing down whether it was tool selection, ordering, or an argument, which is exactly what the trajectory layer is for.

Mocking the Tool vs Exercising It Live

There are two legitimate ways to test a tool call, and they're not interchangeable, they check different things. Mock the tool and assert on the call: fast, no external dependency, and it isolates the agent's decision logic from whether the tool implementation happens to work today. Let the tool run for real and assert on the effect: slower, sometimes requiring test infrastructure, and it's the only way to know the action actually landed.

"""
Assertion type 4: mocking the tool vs exercising it live.

Two legitimate ways to test a tool call, and they check different things.
Mock the tool and assert on the CALL: fast, isolates the agent's decision
logic from the tool's implementation. Let the tool run live and assert on
the EFFECT: slower, but the only way to know the action actually worked.
Neither replaces the other.
"""
from unittest.mock import MagicMock

from agent import Agent
from conftest import ScriptedLLM


def test_mocked_tool_asserts_only_that_the_call_was_made(app_state):
    mock_book_flight = MagicMock(return_value={"booking_id": "BK-MOCK", "status": "confirmed"})
    tools = {"book_flight": mock_book_flight}

    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = Agent(tools=tools, llm_decide=llm)
    agent.run("Book flight FL-100 for Aug 1, A. Rivera")

    mock_book_flight.assert_called_once_with(
        flight_id="FL-100", date="2026-08-01", passenger_name="A. Rivera"
    )
    assert "BK-MOCK" not in app_state.bookings, (
        "a mocked tool never touches real application state; this test proves "
        "only that the agent tried to call it correctly, nothing more"
    )


def test_live_tool_call_asserts_the_booking_actually_exists(make_agent, app_state):
    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for Aug 1, A. Rivera")
    booking_id = trajectory[0].result["booking_id"]

    assert booking_id in app_state.bookings, (
        "letting the tool run live is the only way to assert the effect: a "
        "booking record that actually exists in application state, not just "
        "a well-formed call"
    )
    assert app_state.bookings[booking_id]["passenger_name"] == "A. Rivera"

The mocked test proves the agent tried to book the right thing. The live test proves a booking record with the right passenger name actually exists afterward. Use the mock in tight, fast unit-style loops where you're iterating on the agent's decision logic itself. Use the live call whenever you need to know the action actually changed something, which is most of the time you care about tool calls at all.

What Happens When the Tool Call Fails

Tools fail. Timeouts, rate limits, a downstream service returning a transient 5xx, none of that is exotic, it's Tuesday. What separates a production-ready agent from a demo isn't that its tools never error, it's what the agent does the moment one does: retry, fall back to a different approach, or surface the failure cleanly instead of swallowing it and reporting success anyway.

"""
Assertion type 5: did the agent recover, retry, or fall over when a tool
call errored.

Tools fail: timeouts, rate limits, transient 5xxs. What matters isn't that
the tool never fails, it's what the agent does next. Inject a tool error
and assert the trajectory shows a recovery, not a silent stop or a
swallowed exception the caller never sees.
"""
from conftest import ScriptedLLM


def test_agent_retries_after_a_transient_tool_error(make_agent, app_state):
    app_state.flight_error_countdown = 1  # first call to book_flight raises ToolError

    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for Aug 1, A. Rivera")

    assert len(trajectory) == 2, f"expected one failed attempt and one retry, got {len(trajectory)} steps"
    assert trajectory[0].error == "booking-service-timeout"
    assert trajectory[1].error is None
    assert trajectory[1].result["status"] == "confirmed"


def test_agent_does_not_silently_swallow_a_permanent_failure(make_agent, app_state):
    app_state.flight_error_countdown = 99  # every attempt fails

    llm = ScriptedLLM([
        {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}},
        None,  # the agent gives up after one failed attempt in this script
    ])
    agent = make_agent(llm)

    trajectory = agent.run("Book flight FL-100 for Aug 1, A. Rivera")

    assert trajectory[-1].error is not None, (
        "the trajectory must show the failure explicitly; a test that only "
        "checks 'the agent didn't crash' would miss a booking that silently "
        "never happened"
    )
    assert "FL-100" not in [b["flight_id"] for b in app_state.bookings.values()], (
        "no booking should exist in application state after a permanent failure"
    )

The first test injects exactly one transient failure and confirms the trajectory shows a failed attempt followed by a successful retry, with the booking actually present afterward. The second test injects a permanent failure and confirms the opposite: the trajectory has to show the error explicitly, and no booking should exist in application state. That second assertion catches a specific, ugly failure mode: an agent that catches the exception internally, says something reassuring to the user, and never actually completes the action. A trajectory test that only checks "did it crash" would miss that completely.

Handling Non-Determinism Explicitly

A real tool-calling model, given the identical prompt twice, can produce two different, equally valid trajectories. Treat that like a normal deterministic unit test, one run, exact match, pass or fail, and you get a suite that's either flaky for reasons nobody can explain or so loose it stops catching anything real. Both failure modes come from the same root cause: applying single-run, exact-match thinking to a system that isn't single-run or exact.

The fix has two parts. First, run the scenario N times, not once, and require K of N to pass rather than treating any single failure as a verdict. Second, and this is the part teams get wrong even after adopting K-of-N: assert on the invariant, the tool name and the argument's shape or type, not the literal string. date == '2026-08-01' breaks the moment the model reformats a functionally identical date as 08/01/2026. A check for "is this a date-shaped string" doesn't.

"""
Assertion type 6: handling non-determinism explicitly.

A real model, given the same prompt twice, can produce two different but
equally valid tool calls. Treating that like a deterministic unit test
(one run, exact match, pass or fail) produces a suite that's either flaky
for no diagnosable reason or so loose it stops catching regressions.

The fix: run the same scenario N times, assert on the INVARIANT (tool name,
argument shape) rather than the exact string, and pass if K of N runs
satisfy it. This file simulates a model that mostly gets it right, sometimes
reformats a value, and occasionally reaches for the wrong tool, using a
fixed, reproducible pattern instead of live randomness.
"""
from agent import Agent


class FlakyFakeLLM:
    """Stands in for a real model whose behavior varies run to run. The
    pattern below is fixed and reproducible: out of every 10 calls, 8 pick
    the right tool with the canonical date format, 1 picks the right tool
    with a differently formatted (but still valid) date, and 1 picks the
    wrong tool entirely."""

    def __init__(self, run_index: int):
        self.run_index = run_index
        self.done = False

    def __call__(self, request, trajectory, tool_names):
        if self.done:
            return None
        self.done = True

        position = self.run_index % 10
        if position == 9:
            # wrong tool entirely
            return {"tool": "get_weather", "arguments": {"city": "New York"}}
        if position == 5:
            # right tool, differently formatted (but still valid) date
            return {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "08/01/2026", "passenger_name": "A. Rivera"}}
        return {"tool": "book_flight", "arguments": {"flight_id": "FL-100", "date": "2026-08-01", "passenger_name": "A. Rivera"}}


def _looks_like_a_date(value: str) -> bool:
    return bool(value) and (value.count("-") == 2 or value.count("/") == 2)


def test_booking_scenario_passes_at_least_k_of_n_runs(tools):
    n_runs = 20
    k_required = 17  # ~85% floor, tuned to this simulated model's measured baseline

    passes = 0
    for i in range(n_runs):
        llm = FlakyFakeLLM(run_index=i)
        agent = Agent(tools=tools, llm_decide=llm)
        trajectory = agent.run("Book flight FL-100 for Aug 1 2026, passenger A. Rivera")

        if not trajectory:
            continue

        call = trajectory[0]
        # Assert on the INVARIANT, not the exact string: right tool, and a
        # date-shaped argument, not "date == '2026-08-01'" verbatim.
        right_tool = call.tool_name == "book_flight"
        plausible_date = "date" in call.arguments and _looks_like_a_date(call.arguments["date"])

        if right_tool and plausible_date:
            passes += 1

    assert passes >= k_required, (
        f"only {passes}/{n_runs} runs picked the right tool with a plausible "
        f"date argument (required {k_required}/{n_runs})"
    )


def test_exact_string_assertions_on_arguments_are_brittle():
    """Documents the anti-pattern directly: an exact-match assertion on a
    tool argument breaks the moment the model reformats a value with no
    functional difference, even though the booking would succeed either way."""
    valid_but_differently_formatted_dates = ["2026-08-01", "08/01/2026", "August 1, 2026"]

    def exact_match_assertion(date_value):
        return date_value == "2026-08-01"

    results = [exact_match_assertion(d) for d in valid_but_differently_formatted_dates]

    assert results == [True, False, False], (
        "an exact-match assertion only tolerates one representation of a "
        "functionally identical date; this is why argument assertions should "
        "check shape and invariants, not string equality"
    )

When a scenario in a suite like this fails intermittently, resist the instinct to conclude "the model is unreliable" and move on. Flakiness in a K-of-N eval is a signal about the test, not a verdict on the model: either the assertion is stricter than the tool actually requires, or the tool's description and the prompt are ambiguous enough that two reasonable interpretations exist. Loosen the assertion to the real invariant, or tighten the tool description, then re-run before you touch the pass threshold.

Wiring This Into CI

The deterministic assertions (right tool, right order, right arguments, mocking, failure handling) behave like any other pytest suite: they run on every push, they either pass or they don't, and a red build blocks the merge. The non-determinism suite needs one adjustment: it's still a hard gate, but it's evaluating a pass rate over N runs rather than a single boolean, so it belongs in the same job, not a separate "best-effort" pipeline that nobody actually watches.

name: agent-tool-call-tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  pytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run deterministic trajectory tests
        run: pytest -v --ignore=test_non_determinism.py

      - name: Run non-determinism tests (K-of-N, allowed to be re-run once on flake)
        run: pytest -v test_non_determinism.py
        continue-on-error: false

Keep both in the same required check. A K-of-N gate that lives outside your required checks is a gate nobody enforces, which is functionally the same as not having one. Report the pass rate in the job summary too (17/20, not just a green checkmark), so a slow decline from 20/20 to 17/20 over a few weeks shows up as a trend someone can act on, instead of quietly riding just above the threshold until it doesn't.

Putting the Layers Together

None of the six checks in this guide replace each other, and none of them replace a behavioral check on the app itself. Tool-selection tests catch an agent reaching for the wrong capability. Order tests catch a sequence that's individually correct and collectively wrong. Argument-accuracy tests, the one to prioritize if you only have time for one, catch the failure mode that looks identical to success from a distance. Mocking and live-execution tests answer two different questions about the same call. Failure-handling tests confirm the agent doesn't quietly lose an action when a tool errors. Non-determinism handling keeps all of the above from becoming either flaky noise or a check so loose it stops meaning anything.

Run all six against your own tool-calling agent, wire the deterministic ones and the K-of-N gate into the same required CI check, and you have a trajectory suite that catches real regressions instead of producing green builds that don't mean anything. None of it requires a platform, a new vendor relationship, or a rewrite of how your agent is built: it's pytest, a handful of fixtures, and the discipline to assert on arguments and not just on tool names. The only thing left uncovered is the one this guide has flagged repeatedly, whether the call's effect actually landed where a user would see it, which is a different test, running against a different target, and worth building deliberately rather than assuming your trajectory suite already implies it. Autonoma supplies that last layer by testing the running application after the tool call and maintaining that behavioral coverage as the code changes, so a correct-looking trajectory and a correct user-visible outcome are verified separately.

Frequently Asked Questions

Trajectory evaluation is the practice of capturing the ordered list of tool calls an agent makes (the trajectory: tool names, arguments, and results) and asserting against it after the run. It typically checks three things independently: whether the agent selected the right tool, whether it called tools in the right order, and whether it passed the right arguments. It's a white-box check on the call itself, not on the effect that call had on the running application.

Yes, and it's the most common real-world failure mode. An agent can correctly select book_flight() and still book the wrong date, the wrong recipient, or the wrong quantity if the arguments are off. A tool-selection-only test passes in that case. An argument-accuracy assertion, which checks the actual values passed to the tool, is what catches it.

Run the same scenario N times instead of once, and require K of N runs to pass rather than treating a single failure as a verdict. Assert on the invariant (the tool name, and the shape or type of each argument) rather than an exact string match, since a functionally identical value (a reformatted date, for example) will fail a strict equality check without being wrong. Wire the K-of-N check into the same required CI gate as your deterministic tests, not a separate best-effort pipeline.

Mocking a tool and asserting on the call verifies the agent's decision logic: did it try to call the right tool with the right arguments. It's fast and has no external dependency, but it never touches real application state. Testing a tool live and asserting on the effect verifies the outcome: does a record with the expected values actually exist afterward. Both are legitimate; they answer different questions about the same call.

Inject a tool error deliberately (a mocked timeout or exception) and assert on what the trajectory shows afterward, not just that the process didn't crash. For a transient failure, assert the agent retried and the retry succeeded. For a permanent failure, assert the trajectory shows the error explicitly and that no partial or false-success state exists in the application, since an agent that swallows the exception and reports success anyway is a worse failure than one that visibly errors.

How to Test AI Agents That Take Actions (Tool Calls)

What Is Trajectory Evaluation?

The Harness: A Framework-Neutral Agent Loop You Can Actually Test

Assertion One: Did the Agent Call the Right Tool

Assertion Two: Did It Call Them in the Right Order

Assertion Three: Did It Call Them With the Right Arguments

Right Tool, Right Order, Right Arguments: Still Not the Same as Right Outcome

How Autonoma Verifies What the Tool Call Actually Did

Mocking the Tool vs Exercising It Live

What Happens When the Tool Call Fails

Handling Non-Determinism Explicitly

Wiring This Into CI

Putting the Layers Together

Frequently Asked Questions

What is trajectory evaluation?

Can an agent pick the right tool and still be wrong?

How do you test something non-deterministic, like an LLM's tool choice?

What's the difference between mocking a tool call and testing it live?

How do you test what happens when a tool call fails?

How to Test AI Agents That Take Actions (Tool Calls)

What Is Trajectory Evaluation?

The Harness: A Framework-Neutral Agent Loop You Can Actually Test

Assertion One: Did the Agent Call the Right Tool

Assertion Two: Did It Call Them in the Right Order

Assertion Three: Did It Call Them With the Right Arguments

Right Tool, Right Order, Right Arguments: Still Not the Same as Right Outcome

How Autonoma Verifies What the Tool Call Actually Did

Mocking the Tool vs Exercising It Live

What Happens When the Tool Call Fails

Handling Non-Determinism Explicitly

Wiring This Into CI

Putting the Layers Together

Frequently Asked Questions

What is trajectory evaluation?

Can an agent pick the right tool and still be wrong?

How do you test something non-deterministic, like an LLM's tool choice?

What's the difference between mocking a tool call and testing it live?

How do you test what happens when a tool call fails?

Related articles

Chatbot Automation Testing: Why Assertions Fail

Ghost Inspector Alternative: Recorder, Framework, or AI?

How to Test the Auth Code an AI Agent Wrote

Why AI Code Review Misses Auth Bugs