Build an evaluation set for a business AI system
A successful demonstration says little about ordinary, ambiguous or difficult requests. A small, carefully designed test set makes quality observable and helps catch regressions after a change.

Where to start
Collect anonymized real cases, add edge cases and requests the AI should decline. Define an acceptable output, then compare versions with the same rubric and human review.
- Representative and difficult cases
- Observable criteria
- Regular comparison
Build a reference set in six steps

AI-generated illustration 1. Bound the task
List permitted tasks, users, response formats and the most costly errors. Sourced research needs different measures from customer support.

AI-generated illustration 2. Collect examples
Use common requests and rare but important cases. Remove personal data and keep the origin and date of each example.

AI-generated illustration 3. Add adverse cases
Include ambiguous requests, conflicting documents, empty inputs, different languages and malicious instructions where these situations can occur.

AI-generated illustration 4. Write a rubric
Score accuracy, sources, format, appropriate refusal and correction time. Define observable levels rather than relying on a general impression.

Editorial illustration 5. Establish a baseline
Run every case on the current version. Ask two people to review a sample and resolve scoring disagreements before comparing a new version.

AI-generated illustration 6. Track regressions
Rerun the set after changing a model, prompt, corpus or tool. Add real incidents and retire outdated examples.
Put the method to work

Practical case
Assemble twelve anonymised cases: six common, three ambiguous, two critical and one the system should decline.

Evidence to keep
For each case, define expected outcome, observable rubric, tested version and any disagreement between reviewers.

Make the decision
Compare versions on this stable set; then add real incidents without retrospectively rewriting baseline results.
Four qualities of a useful set

Representative
Cases reflect real uses and their frequency.

Risk coverage
Costly errors and expected refusals are included.

Repeatable
Inputs, versions and scoring rules are retained.

Current
Incidents and business changes feed the set.
Compare services on identical cases
Shortlist services suited to the task; the test set reveals differences in your own context.
OpenAI Platform
model APIs
OpenAI · US
Visit official siteClaude Platform
model APIs
Anthropic · US
Visit official siteGoogle Antigravity
coding agent
Google · US
Visit official siteMagika
file type identification
How is this selection produced?
These profiles are linked to the guide’s subject in our editorial catalogue. Inclusion is neither a ranking nor a performance measurement on your documents. Methodology.
Related tool families
Frequently asked questions
How many cases are needed?
Start with a small, diverse and stable set, then add incidents and missing cases. Coverage matters more than an arbitrary count.
Is automatic scoring enough?
No. For important outcomes, review a sample manually and check disagreement between evaluators.
When should tests be rerun?
After changes to the model, instructions, data or tools, and regularly for critical cases.
The references below expand on the concepts and checks discussed. Scenarios and trial frameworks remain editorial proposals; provider documentation describes its own product rather than an independent benchmark.
Official sources
Does a search retrieve the right documents?
Evaluate retrieval before judging the generated answer. In a bounded corpus, identify relevant documents for a question, then count relevant hits, irrelevant hits and missed relevant documents. Count each document once and use the same maximum result count when comparing two systems.
Calculation runs locally in your browser without storage. Without JavaScript, use the formulas below; displayed values remain those of the example.
- Precision: avoid noise
- Relevant retrieved ÷ all retrieved documents. Here: 3 ÷ (3 + 1) = 75%.
- Recall: avoid omissions
- Relevant retrieved ÷ all relevant documents in the corpus. Here: 3 ÷ (3 + 2) = 60%.
- F1
- 2 × relevant retrieved ÷ (2 × relevant retrieved + irrelevant retrieved + relevant missed). Here: 6 ÷ 9 ≈ 66.7%.



A zero denominator produces “undefined”. Without an inventory of relevant documents, recall is unknown: do not infer it across the entire web. These measures assess neither answer truthfulness nor the severity of a missed document; omitting a decisive exception remains a blocking error.



