How do you test a Generative AI system?
With a golden dataset of realistic inputs and expected outputs, written acceptance criteria, automated scoring for the properties that can be scored, calibrated human review for the ones that cannot, safety and adversarial testing, and the same checks running on sampled production traffic after release.
Unit tests still apply to the software around the model. They cannot tell you whether the model's answer was right, which is what the rest of this page is for.
How do you evaluate RAG?
In two stages. First retrieval: for each question in the evaluation set, were the right passages retrieved, scored for precision, recall and rank. Then generation: given the retrieved context, was the answer correct, grounded in that context and cited. Scoring them separately is what lets you fix the right component.
How do you measure hallucination?
By decomposing each answer into claims and checking whether every claim is supported by the retrieved context (groundedness) and, where a reference exists, whether it is true (factuality). The result is a rate per answer and per dataset, judged automatically and verified by human sampling. We do not promise a fixed hallucination percentage; we measure it for the specific use case.
How do you evaluate an AI agent?
With a set of tasks that have known correct outcomes, run in a simulated environment that returns realistic tool responses including failures, scoring task success, tool-call correctness, steps taken, unsafe actions and whether the agent escalated when it should have. Production traces are then sampled and scored the same way.
What is an AI golden dataset?
A curated set of inputs with agreed correct outputs or rubric scores, drawn from real usage and deliberately extended with edge cases and adversarial examples. It is the fixed reference every release is measured against, and it grows from production feedback.
Building it is a joint effort with the business, because deciding what a correct output is requires the people who do the job.
Can LLM-as-a-judge be trusted?
Partly, and only when calibrated. A model can score outputs against a rubric at scale, but its agreement with human reviewers has to be measured on the same samples, its known biases (toward length, toward its own style) controlled, and its judgments spot-checked continuously. Used that way it is a useful instrument; used blindly it is a second opinion from the system under test.
What should be human evaluated?
Anything where correctness needs domain judgment, where the cost of a wrong output is high, where the rubric is new or contested, and a rolling sample of everything else to keep the automated judge honest. Human evaluation is expensive, so it is aimed where automation is least reliable.
What should be automated?
Everything with a checkable property: retrieval metrics, schema and format compliance, tool-call correctness, safety classifiers, regression against the golden dataset, latency and cost. Automation is what makes evaluation a release gate rather than a quarterly review.
How do you test AI before production?
By writing the acceptance criteria first, building the golden dataset with the business, running automated and human evaluation on every candidate release, red teaming the system for unsafe outputs and actions, and rehearsing the failure and rollback paths. A system that passes ships with monitoring that keeps running the same tests.
How the operating layer runs these checks after launch