"It seems better" is not a test plan.
Ask most teams how they know their AI system works and you'll get a demo, a vibe, and a prayer. Nobody would ship ordinary software without tests, yet LLM systems — which fail probabilistically and regress silently — routinely ship on eyeball QA. I build evaluation harnesses: golden test sets, automated scoring, regression gates, and production monitoring that turn 'seems better' into a number that moves.
Every change to an LLM system — a prompt tweak, a model upgrade, a new retrieval index — changes behavior somewhere you didn't look. Vendors swap model versions under you. A fix for one failure quietly breaks three other cases. Without evals, you find out from users; with them, you find out in CI. That's the entire difference between operating a system and gambling on one.
An eval harness has unglamorous parts and they all matter: a golden test set built from your real cases including the nasty ones, scoring that combines deterministic checks with calibrated LLM-as-judge grading, and gates wired into your pipeline so a quality drop blocks the release. Then production monitoring samples live traffic against the same rubric, so drift shows up on a dashboard instead of in a complaint.
What gets measured
- Task accuracy — correctness against ground truth: right answer, right extraction, right route, right citation.
- Grounding and faithfulness — whether answers stay inside retrieved sources, and whether citations actually support the claims they decorate.
- Safety and containment — injection resistance, refusal correctness, tone, and policy compliance, tested adversarially and on schedule.
You wouldn't run finance on unaudited numbers. Don't run your AI on unaudited answers.
An eval harness in four parts.
Questions, answered.
What is an LLM eval, in plain terms?
A repeatable test that runs your AI system against known cases and scores the outputs — like unit tests, but for probabilistic behavior. A good eval suite tells you within minutes whether a change made the system better, worse, or differently broken, and on exactly which cases.
How do you evaluate outputs that have no single right answer?
With rubrics and calibrated judges. Deterministic checks handle format, facts, and citations; an LLM-as-judge grades qualities like helpfulness and faithfulness against a written rubric — and the judge itself is validated against human graders until they agree, so you are not grading vibes with vibes.
How big does a golden test set need to be?
Useful starts at 50–100 well-chosen cases; mature suites grow into the hundreds by harvesting production failures. Coverage of your real case distribution — including the ugly tail — matters far more than raw count. Every production incident should become a permanent test case.
Can evals run on a system another vendor built?
Yes, and it is one of the highest-leverage uses — an independent eval harness around a vendor system tells you what you are actually getting before renewal, and gates their updates the same way it would gate your own. Vendors behave differently when the scoreboard is not theirs.
Put a number on "does it work?"
If your AI quality process is a demo and a shrug, let's fix that before your next model upgrade does it for you.
