Field Notes · Model Evaluation

Benchmark the Agent, Not Just the Model

By Will Hess · Founder, Inside Intelligence

Before you let an agent take over a real job — answer a phone, parse a radio call, draft a document that leaves your building — you have to know what you're actually working with. Not what the marketing copy claims. Not what a single cherry-picked demo shows. What the model can do under the exact conditions you'll put it in.

That's a different exercise than a leaderboard, and it's worth doing deliberately.

The trap: a leaderboard tells you almost nothing

Public model scores are measured on benchmark questions the model may have effectively memorized, under ideal sampling and unlimited time. They answer a different question than the one you actually care about, which is: will this model do the work reliably, the way I need it done, at the speed I need it done?

The rule

A benchmark ranks models against each other. A work evaluation tells you whether one model can do your job. Run your own test.

The method: a fixed, repeated, scored suite

We evaluate a candidate against a job by defining a suite of tasks that mirror the real work — then running every candidate through the same suite, under the same conditions, and scoring each answer against ground truth. Every candidate answers the exact same prompts. Nothing is skipped, nothing is improvised mid-run.

Three decisions make this worth doing:

The part nobody talks about: the hardware is part of the test

Models run on something, and that something shapes what you observe. On a shared or constrained GPU, two models fighting for the same memory produce timings that are meaningless. A large model that spills past video memory runs at a fraction of the speed of one that fits. If you're not controlling for that, your speed comparison is fake.

Two practices keep the measurement clean:

What we actually find, run after run

The pattern repeats: the newest, fastest candidate almost always keeps up on correctness — and then reveals its character on one specific leg. Different builds handle the deliberately ambiguous case differently. Some refuse. Some over-think until they time out. Some answer cleanly because they don't notice the trap at all.

The one that catches it — that names the mechanism, walks the numbers, and flags the ambiguity — is the one you trust with real work. And the one that times out on that same leg is the one you don't, no matter how fast it looked elsewhere.

The counterintuitive bit

The model that looks best on the leaderboard is frequently not the model that wins your own fixed suite. It's the fit — model, quantization, sampling, hardware — that decides, and only your own test measures that.

The guardrail

The guardrail is the suite itself. It's a fixed fixture we re-run whenever we consider a change — a new model, a re-tuned quantization, a different card. Baseline it, then only change one variable at a time. If the numbers shift, you know what moved them.

The job doesn't change because a shiny new model shipped. The job changes only when your own testing says the new model does it measurably better.

The takeaway

The most capable-sounding model isn't the one you can ship. It's the one you validated — on your tasks, at your conditions, at your speed, scored for being right. The discipline of running that fixed, repeated, honestly-scored suite is what separates an agent you trust from a demo you're nervous about.

Benchmark the agent, not just the model — and let your own test, not the marketing, decide what gets a job.


← Back to Field Notes