How Do You Value an AI Agent? The Case for Agency Evaluation
Benchmark scores measure what an agent knows. Agency scope measures what you can trust it to do without watching. These are not the same number, and the second one matters more.
We built a trading agent that makes buy decisions on a $200 allocation. Max position size $50. Daily loss cap $10. Limit orders only. No margin, no options. Every trade requires Discord approval — a message with the rationale, the ticker, the size, and two buttons: approve or reject.
The agent is not maximizing for returns. It is maximizing for something more useful: a track record. Each approved trade, executed correctly, expands the scope of what we trust it to do autonomously. Each failure — wrong ticker, wrong logic, missing a mandate — contracts it.
This is agency evaluation in practice. Not a benchmark score. Not a capability claim. A specific, measured, incrementally expanding authorization surface. The agent starts with almost no autonomy and earns its way toward more.
It turns out this is a useful way to think about AI agents more generally.
Why Benchmark Scores Are the Wrong Signal
Benchmarks answer: can this agent do X in a controlled environment? SWE-bench, HumanEval, MMLU — these are knowledge tests. They evaluate the agent against a fixed, curated task distribution with known correct answers.
That tells you almost nothing about what the agent will do in your specific environment, on your specific tasks, when it encounters ambiguity at 2 AM on a Friday and you are not available to supervise.
The benchmark worldview treats AI agents like exam takers. Measure their knowledge, infer their capability, deploy them. But agents are not exam takers. They are operators in an environment that the benchmark designers did not model. The relevant question is not "how smart is this agent?" — it is "what can I safely let this agent do, in this context, without watching?"
These are different questions with different answers, and confusing them is how you get agents that ace benchmarks and then make expensive mistakes in production.
Agency Is a Spectrum, Not a Switch
Agency is not binary. It exists on a spectrum from fully supervised — the agent proposes, a human approves every action before execution — to fully autonomous, where the agent acts and the human reviews results after the fact.
Most production systems that work well live somewhere in between, at different positions for different action classes. Our trading agent needs approval for every trade. Our browser automation runs unattended but flags when it encounters an unexpected state. Our memory system returns results without approval but refuses to surface claims it cannot trace to a source.
The right position on the spectrum depends on three things: the task, the stakes, and the track record. A new employee gets more supervision than an experienced one doing the same work. The same agent executing a routine task gets more autonomy than when attempting something it has never done. The same action gets more autonomy after 50 successful executions than after zero.
Agency evaluation measures where an agent is on this spectrum for each action class it might take — and how that position changes as evidence accumulates.
The Gate Model
The mechanism we use for every significant agent action: an evaluation gate. The gate asks four questions before allowing an action to execute:
The gate outcome is one of four: proceed autonomously, require approval, require explanation, or block. Which outcome the gate returns depends on the action class, the current stakes, and the agent's track record on that class of action.
Importantly, the gate is not static. An agent that has executed a class of action 50 times with zero mandate violations earns more latitude than one attempting it for the first time. The gate expands with demonstrated reliability and contracts when failures occur. This is how you build trust incrementally rather than granting it upfront based on a benchmark score.
Agency Evaluation Is Already in Your Stack — You Just Do Not Track It
When we looked at what we had already built, we realized agency evaluation was implicit everywhere. We just had not been measuring it explicitly.
The selector ladder in our browser automation is an agency evaluation: each level of the hierarchy represents a different confidence level for a selector, and the automation's behavior changes based on which level it is operating at. High-reliability ARIA selectors run unattended. CDP coordinate clicks — the lowest reliability level — always trigger a log and a flag.
The memory retrieval gate is an agency evaluation: the system returns a result only if it can trace that result to a source. Fabrication count must stay at zero. A retrieval that cannot ground its output is blocked, not surfaced with a disclaimer.
The eval harness itself — 39 gold queries, a semantic judge, three-number measurement — is an agency evaluation for the retrieval system. It tracks whether the system has earned the right to expand its retrieval scope. Completeness improvements are approved. Fabrication regressions are immediate rollbacks.
In each case, the pattern is the same: every action class has a reliability threshold, a measurement method, and a gate that enforces the threshold. The threshold adjusts as evidence accumulates.
What This Means for Valuing AI Businesses
There is a straightforward implication for how AI agent companies should be evaluated — by investors, by customers, by anyone making a deployment decision.
When a company claims its agent can "autonomously handle X", the useful follow-up is not "what is its benchmark score on X?" — it is: what is its agency scope for X? Specifically:
- —For which actions has the agent demonstrated reliability sufficient for unattended execution?
- —What is the failure mode when the gate fires? Does the system stop, escalate, or continue?
- —What evidence is used to calibrate the gate? Track record in production, or performance in a controlled evaluation?
- —When the agent encounters a situation outside its calibrated scope, what happens?
An agent with a broad, well-calibrated agency scope — one that has demonstrated reliability across high-stakes actions in its actual deployment environment — is worth more than one with a strong benchmark score and no production track record. Agency scope, measured against real task classes in real environments, is the number that matters for deployment decisions.
We think this framing will become standard for AI due diligence. The question will shift from "show me your benchmark scores" to "show me your agency scope and the evidence that supports it."
What We Are Building Toward
Right now we have implicit agency evaluation in three places: browser automation (selector ladder), memory retrieval (fabrication gate), and trading (Discord approval with track record tracking). These are not yet unified. Each has its own threshold logic, its own measurement method, its own escalation path.
The next step is a unified agency evaluation framework — a single system that tracks, per action class, the reliability track record, the current gate threshold, the measurement method, and the gate outcome history. Not a dashboard. Infrastructure. The same interface used to configure what any action class requires before execution.
If you are deploying agents in production and thinking about how to build this kind of trust — or if you are an investor trying to evaluate agent companies on something more meaningful than benchmark scores — we want to compare notes. rahul@ollie-labs.com