Agent Evals Are Not About Answers. They Are About Work.

Most eval conversations still inherit the shape of classroom grading: give the model a prompt, compare the answer to a reference, assign a score.

That is useful for some things. It is not enough for agents.

Agents do not merely answer. They search, click, query, write, update, purchase, schedule, deploy, delete, retry, ask for help, and sometimes quietly misunderstand the entire job. Once a system has tools and autonomy, the unit of evaluation is no longer the response. It is the episode.

The first principle is simple:

Agentic eval =
did the right world change happen,
by an acceptable path,
without hidden damage,
often enough to trust?

The RDS failure

Imagine a user asks an AWS cost agent: "How much did we spend on databases last month?"

The agent queries Cost Explorer for Amazon Relational Database Service, gets $40,000, and answers: "You spent $40,000 on databases last month."

From inside the trace, this can look clean. The API call succeeded. The number came from a real source. The final answer is faithful to the returned result. No hallucination. No crash. No obvious contradiction.

It is still wrong.

"Databases" might include RDS, DynamoDB, ElastiCache, Redshift, OpenSearch, and other storage-adjacent services depending on the user's business context. The failure is not that the agent fabricated a number. The failure is that it collapsed a semantic category into the first service name that matched.

This is why agent evals are hard. The most dangerous failures often look like success from inside the trajectory.

Finished is not successful

A browser agent clicks Submit and sees the page change. A coding agent edits a file and says the task is done. A sales agent drafts an email and moves on. A FinOps agent retrieves a cost number and summarizes it.

In each case, the agent may have finished its visible sequence of actions without satisfying the actual goal.

clicked != submitted
submitted != accepted
drafted != sent
queried != understood
retrieved != complete
stopped != successful

This is the core shift from output evals to environment evals. You do not only inspect what the agent said. You inspect what changed.

Did the calendar event exist? Did the payment process once, not twice? Did the database row change correctly? Did unrelated records remain untouched? Did the email actually send? Did the code compile? Did the user end up better off?

Outcome is not enough

Outcome evals matter. If the requested state did not happen, the agent failed. But outcome alone is a weak teacher.

A bad process can get lucky. A good process can fail because the environment changed. A reckless agent can reach the right final answer while leaking data, ignoring approvals, retrying expensive calls, or corrupting unrelated state. If the reward function only looks at the final answer, it will eventually teach the agent to chase the wrong thing.

This is where process supervision becomes essential. Not because process is morally nicer. Because process contains information the final outcome erases.

outcome_eval:
  did the final state match the goal?

process_eval:
  did the agent gather enough evidence?
  did it preserve scope?
  did it use tools correctly?
  did it recover from weak evidence?
  did it know when to stop or escalate?

side_effect_eval:
  what else changed?
  was anything exposed, duplicated, deleted, or mutated?

reliability_eval:
  does this work repeatedly under realistic conditions?

There is an old line from the Gita that maps surprisingly well here: you have a right to the work, not to the fruits. For agents, the practical version is: do not let the reward function corrupt the policy. Evaluate the quality of action under uncertainty, not only the terminal prize.

Reference-free evals hit a wall

Reference-free evals are attractive because they can run from the trace alone. They can catch invalid tool parameters, schema errors, arithmetic mistakes, contradictions, unsupported claims, ignored failures, and answers that do not follow from evidence.

They should be part of every serious agent system.

But they cannot reliably catch what never entered the trace. Omission is the brutal case. If the agent never searched for DynamoDB, a trace-only evaluator may not know that DynamoDB was missing. If the agent misunderstood "database" as "RDS," the trace may be internally consistent and externally incomplete.

The rule:

You can only catch failures that leave evidence.

That means reference-free evals need reference-guided companions: domain maps, policy constraints, user-specific expectations, historical corrections, golden tasks, and human review for the cases where absence is the bug.

A practical framework

A usable agentic eval has five parts:

1. Task distribution. What work is the agent expected to perform, and how representative are the tasks?
2. Environment. What initial state, tools, permissions, latency, failures, and policies surround the task?
3. Trace. What did the agent see, decide, call, observe, claim, and change?
4. Graders. Which parts can be checked deterministically, which need an LLM judge, and which require human or domain review?
5. Decision policy. What happens after the eval: ship, block, retry, ask a human, collect more traces, or add a regression test?

The last part is the one teams skip. An eval with no decision attached is usually dashboard theater. A real eval changes what the system is allowed to do next.

Build from failures, not vibes

The best eval work starts with traces, not taxonomies. Sample real runs. Read them like an operator. Find the first upstream failure, not every downstream symptom. Then turn recurring failures into narrow checks.

sample traces
-> label first failure
-> cluster failure modes
-> write one narrow evaluator per cluster
-> calibrate against human judgment
-> promote fixed bugs into regression cases
-> monitor drift in production

Do not start with "quality: 4.2." Start with "agent answered a database-cost question using only RDS." That is concrete. It can become a check. It can become a domain rule. It can become a regression case. It can make the system better next week.

Reliability is multiplicative

Agents are loops, so average quality is a trap. A task with six steps needs all six to work. If each step is 95% reliable, the full run is not 95% reliable. It is roughly 74% reliable before you count tool failures, environment drift, and ambiguity.

This is why pass@k and pass^k describe different worlds. Pass@k asks whether the agent can solve the task if given several tries. Pass^k asks whether it can keep succeeding across repeated attempts. Demos love the first. Operations need the second.

Autonomy should be earned from repeated clean behavior in realistic environments, not from one lucky transcript.

The first principle for the series

Evals are not a scoreboard for agents. They are the discipline of converting messy work into observable evidence of trust.

That means the center of gravity moves from answers to episodes, from benchmarks to environments, from outcomes to process, from scores to decisions, and from one-time grading to a compounding loop where every correction becomes a permanent check.

The rest of this series will go deeper into that loop: how to design task environments, how to grade trajectories, when to trust LLM judges, why omissions are the hardest failures, how to measure reliability, and how evals should govern agent autonomy.