Agent Evaluation for Tool Use: Why pass@1 Lies, and How to Measure Reliability

Here is the number that should change how you evaluate a tool-using agent. Reliability under repetition decays exponentially: an agent that succeeds 90% of the time on a single attempt succeeds on all five of five independent attempts only 59% of the time, and on all eight only 43%. The demo that worked, the benchmark that scored 90%, the screencast that closed the deal: none of them measured the thing that breaks in production, which is not whether the agent can do the task but whether it does it every time. A single-attempt success rate is the wrong metric for a system that will run the same task thousands of times, and most agent evaluation reports exactly that wrong metric.

This is the agents-in-production problem stated as a measurement problem. An agent pointed at a real task deserves the same eval discipline as any model deployment, and tool use is where the gap between "it worked in the demo" and "it works at scale" is widest. This piece is about what to measure, the benchmarks that measure it, and how to build a harness that tells you the truth.

Why tool-use evaluation is a different problem

Evaluating a chat response asks one question: is this text good? Evaluating a tool-using agent asks several, and none of them is about text. The agent's output is a trajectory of actions against a stateful environment: it calls tools (each defined by a schema, whether an MCP server or a function spec), passes arguments, reads results, calls more tools, and eventually changes the world (books the flight, updates the record, sends the message). Correctness is not whether the final message reads well; it is whether the right actions happened, in the right order, with the right arguments, and whether any wrong, irreversible action happened along the way.

That makes four distinct things to measure, which a single score conflates:

• Tool selection: did it call the right tool, and just as important, did it refrain from calling one when none applied? Hallucinated tool calls (inventing a function or calling an irrelevant one) are a major failure mode that pure accuracy hides.
• Argument accuracy: were the parameters correct, well-typed, and grounded in the actual request, not confabulated? Note that constrained decoding guarantees a tool call's arguments are schema-valid, never that they are correct, so this dimension must be scored on the values, not on whether the call parsed.
• End-state correctness: after the whole run, is the environment in the state the user wanted? This is the outcome that actually matters.
• Safety of the path: how many wrong or irreversible actions did it take getting there? An agent that reaches the right end-state after also deleting the wrong record is not a success.

A benchmark that reports one accuracy number has collapsed these, and the collapse hides exactly the failures that cost money in production.

The metric that matters: pass^k, not pass@1

The single most important shift in agent evaluation is from pass@k to pass^k. They look similar and mean opposite things. pass@k asks whether at least one of k attempts succeeded, which rewards a lucky run and is the right metric when you can retry cheaply and verify the result. pass^k, introduced by τ-bench, asks whether all k attempts succeeded, which measures consistency, and consistency is what an unsupervised production agent needs.

The two diverge brutally because pass^k is p^k. An agent at 90% single-attempt success looks strong on a leaderboard and is only 59% reliable over five tries and 43% over eight. An agent you would describe as "works almost always" fails one time in ten, and over a day of real traffic that one-in-ten is a stream of incidents. Reporting pass@1 for a system that runs continuously is not just optimistic, it measures a property (can it ever succeed) that is irrelevant to the deployment question (will it succeed this time, and the next, and the next). If you change one thing about how you evaluate agents, report pass^k.

The benchmarks, and what each actually measures

Three families of benchmark cover the space, and they are not interchangeable.

BFCL (the Berkeley Function-Calling Leaderboard) measures function-calling correctness in isolation. Its most useful properties are that it evaluates by state, checking the actual state of the API system after execution rather than string-matching the call, and that it explicitly tests relevance detection, whether a model correctly declines to call a function when none fits. Its V3 added multi-turn and multi-step tasks across domains like vehicle control, trading, travel booking, and file systems, with state-based verification of both the end state and the execution path. BFCL is the right tool for "can this model call functions correctly," the component-level question.

τ-bench and τ²-bench (Sierra and Princeton) measure the harder, system-level question: can the agent hold a multi-turn conversation with a simulated user, follow domain policy rules, and manipulate a database correctly. τ-bench pairs the agent with a user simulator and a rule-bound environment in airline and retail domains, and it is where pass^k originated. τ²-bench raised the difficulty with a dual-control setting, where both the agent and the user can act on a shared, changing environment, modeling tasks like telecom support where the agent has to guide a user who is also taking actions. The headline finding is sobering: moving from solo control to dual control dropped single-attempt success by as much as 40%, because coordinating with an agent you do not fully control is far harder than acting alone.

The lesson from holding these side by side: a model can top BFCL (it calls functions correctly) and still fail τ²-bench (it cannot reliably drive a real multi-turn interaction), because they measure different layers. Component competence does not compose into system reliability for free.

How to evaluate: three methods, ranked

Once you know what to measure, there is the question of how to score a run, and the three available methods are not equally trustworthy.

State-based, verifiable evaluation is the gold standard. Define the correct end-state of the environment (this record updated, that booking created, this message not sent) and assert it programmatically after the run. It is deterministic, unbiased, and cheap to run thousands of times, which is what pass^k requires. This is why the strongest agent benchmarks are built as environments with verifiable end-states rather than as transcripts to be graded. Whenever your task has a checkable outcome, this is the method to use.

Trajectory evaluation scores the path, not just the destination: the sequence of tool calls, their arguments, the intermediate reasoning, the retries. It catches an agent that reached the right end-state by a dangerous or wasteful route, which end-state checking alone misses. It is more work to build and more brittle, but it is the only way to measure the safety-of-path dimension directly.

LLM-as-judge is the flexible, least-trustworthy option, and it needs the same skepticism the evaluation crisis demands everywhere. Using a model to grade an agent's trajectory scales and handles fuzzy criteria, but it carries the documented judge biases (position, length, agreeableness) and, on complex agentic evaluations, error rates above 50% and agreement with domain experts only in the 64 to 68% range. That is a weak signal, not ground truth. Use it for triage and for criteria you genuinely cannot verify programmatically, always calibrated against a human-labeled slice, and never as the sole arbiter of whether an agent works.

The ranking is the practical takeaway: verify state where you can, score trajectories where safety matters, and treat an LLM judge as a noisy assistant, not a referee.

The benchmarks are gameable too

One more warning, because it is the eval-crisis lesson applied to agents. Public agent benchmarks are targets, and targets get gamed. Researchers at Berkeley's RDI showed that major agentic benchmarks, including GAIA, can be exploited to reach near-perfect scores without actually solving the underlying tasks, by exploiting structure in the harness rather than demonstrating capability. A leaderboard position on an agent benchmark is therefore subject to the same discount as any other: it is a hypothesis about capability, not proof of it, and it tells you little about your agent on your tools and your tasks.

The conclusion is the same one the whole eval beat keeps reaching: the number that matters is the one you measure on your own environment, with your own verifiable end-states, run enough times to see the tail.

Measure cost, not just correctness

Reliability is necessary but not sufficient, because two agents that both reach the right end-state can differ by an order of magnitude in what they spend getting there. One solves the task in four tool calls and a few thousand tokens; the other flails through forty calls, re-reading the same data and retrying, burning ten times the tokens and latency for the same outcome. On a pass^k scoreboard they tie. In production they do not.

A complete harness therefore reports efficiency alongside correctness: tool calls per task, tokens per task, wall-clock latency, and the rate of redundant or repeated calls. These are not secondary niceties; at scale they are the cost structure of the whole system, and a reliable-but-wasteful agent can be more expensive than a less reliable one you simply retry. Watch the distribution, not the mean, because the tail task that took 80 calls is where both the cost and the latency SLO break. Rising tool-call counts across a release are also an early warning: the agent is losing the plot in a way the pass^k number has not caught yet.

Building a harness that tells the truth

A useful agent eval harness is not exotic. It is a few disciplined choices:

• Define tasks by verifiable end-state. For each task, write an assertion over the environment after the run (database rows, API state, files), not a check on the chat transcript. If a task has no checkable outcome, it is not yet a test, it is a vibe.
• Simulate the user for multi-turn tasks. Real agent failures happen across turns, so a fixed prompt is not enough. A scripted or model-driven user simulator, as τ-bench uses, exposes the multi-turn breakdowns a single-shot test cannot.
• Run k times and report pass^k. Pick a k that reflects your reliability needs (production-critical flows want k of 8 or more) and report the all-pass rate, not the any-pass rate. The gap between them is your reliability debt.
• Separate the dimensions. Log tool selection, argument accuracy, end-state, and irreversible actions separately. A single pass/fail hides which of the four broke, and you cannot fix what you cannot localize.
• Count irreversible wrong actions explicitly. Final-state success does not capture damage done en route. Track it, because in production the cost of an agent that succeeds and also takes one irreversible wrong action is not discounted by its success rate.

Concretely, the core loop is small:

def eval_task(agent, task, k=8):
    runs = []
    for _ in range(k):
        env = task.fresh_environment()              # reset state every run
        agent.run(task.prompt, env, user=task.user_simulator())
        runs.append({
            "passed": task.check_end_state(env),    # assert env state, not transcript
            "tool_calls": env.tool_call_count,
            "irreversible_errors": env.count_irreversible_wrong_actions(),
            "tokens": env.tokens_used,
        })
    return {"pass_hat_k": all(r["passed"] for r in runs), "runs": runs}

The shape is the whole point: a fresh environment per run, a state assertion rather than a transcript grade, k repetitions to get pass^k, and the cost and safety counters logged next to the boolean. Everything else is task-specific.

Do this and you get the honest picture: not whether your agent can do the task, but how often it does it correctly, where it breaks when it does, and what it costs when it goes wrong. That is the difference between an agent you can deploy unsupervised and a demo you have to babysit, and the only way to know which one you have is to measure the right thing.

Key Takeaways

1. pass@1 is the wrong metric for a production agent. Reliability decays as p^k: a 90%-success agent is only 59% reliable over five attempts and 43% over eight. Report pass^k (all k succeed), not pass@1 or pass@k.
2. Tool-use output is a trajectory of actions, not text. Measure four separate things: tool selection (including correctly not calling), argument accuracy, end-state correctness, and the safety of the path. A single score hides which broke.
3. Hallucinated and irrelevant tool calls are a primary failure mode. Evaluate relevance detection, whether the agent declines to call a tool when none fits, not just accuracy when it does call.
4. The benchmarks measure different layers. BFCL scores function-calling correctness in isolation with state-based checks and relevance; τ-bench and τ²-bench score multi-turn, policy-bound, user-facing agent reliability. Topping one does not predict the other.
5. Dual control is much harder than solo. τ²-bench showed single-attempt success dropping up to 40% when the agent must coordinate with a user who also acts. Coordinating is harder than acting alone, and most real tasks are dual control.
6. Rank your scoring methods. Verifiable state-based evaluation is the deterministic gold standard; trajectory evaluation catches dangerous paths; LLM-as-judge is a biased, error-prone signal (over 50% error on complex tasks) to use only calibrated and never alone.
7. Agent benchmarks are gameable. Major ones including GAIA were shown exploitable to near-perfect scores without solving the tasks. A leaderboard number is a hypothesis, not proof; measure on your own environment.
8. Build the harness around verifiable end-states. Assert environment state (not transcripts), simulate the user across turns, run k times for pass^k, separate the four dimensions, and count irreversible actions explicitly.

The Acing AI newsletter covers agent evaluation the way this piece does: the metric that survives contact with production, not the one that wins the demo. Subscribe for the measured take.