AI EVALUATION
A demo is not deployment: use evals to define acceptance
One impressive answer proves that the model may do the task. Evals turn real inputs, exceptions and system changes into standards the team can inspect before launch.
- 01Specify Ideal result, unacceptable failure, exception rule
- 02Measure Test continuously on representative work
- 03Improve Improve by failure type, then rerun the same standard
- Golden Set
- Real, boundary and high-risk cases
- Review
- Rules plus calibrated human review
- Gate
- Separate capability and regression
1. Demo vs Eval｜Possibility is not readiness to launch
A carefully chosen input, clear prompt and experienced presenter can produce an impressive result. Real work includes scanning errors, missing fields, conflicting rules, strange formats, time pressure and a small number of cases that cannot be wrong.
A demo should answer whether the idea deserves exploration. An eval should answer what must be true before the company buys, launches or expands it. They serve different decisions and should not share one success bar.
2. Acceptance Loop｜Specify → Measure → Improve
Specify turns 'be accurate' into ideal outputs, unacceptable errors and exception rules. Measure tests representative work continuously. Improve changes context, tools, workflow or model in response to failure patterns, then reruns the same standard.
OpenAI describes contextual evals as a way to turn fuzzy business objectives into measurement. The durable principle is not a particular platform. Leadership defines what 'worth buying' means in conditions a team and system can inspect.
3. Golden Set｜Build acceptance cases from real work
Ask domain experts for normal, incomplete, ambiguous, boundary and high-risk examples, then record acceptable outcomes. The first set does not need to be enormous, but it must represent the cases that shape purchase and adoption confidence.
A golden set is not a finished exam. When production reveals a new exception, a reviewer makes a major edit or a model changes, representative cases enter the set. Over time, the company accumulates its own quality asset.
4. Scorecard｜Measure the result, path and exception
The final text can hide the real risk. Whether an agent cited the right source, used an approved tool, stopped when required, escalated to the right person and made failure visible are also dimensions of product quality.
A useful leadership dashboard includes task completion, reviewer edits, undetected errors, exception volume, escalation quality, cycle time and adoption—not one average 'AI accuracy' number.
5. Capability / Regression｜Test progress and backsliding separately
Capability evals ask whether the new system can handle harder work. Regression evals make sure that changing a prompt, context, model or tool did not quietly break what already worked. Without both, iteration becomes a cycle of fixing one failure and creating another.
Anthropic notes that multi-step tool use makes agent evaluation harder and no single score explains all behaviour. Automated graders, deterministic checks and calibrated human review need to work together.
6. Pilot Checklist｜Start with recent cases
Do not begin by asking the team for a complete AI specification. Bring a small set of recent real examples and mark the ideal result, unacceptable failure, who may decide and how long completion takes today.
If those examples still cannot be made explicit, the most useful thing to improve is not the model but the acceptance standard.
Limitations and scope
Evaluation-set size, graders and pass thresholds must match the risk of the task. A small set of examples is a discovery starting point, not a deployment sample requirement. LLM graders need human calibration and are not objective truth or a substitute for sector expertise.