11 – Evaluation Basics¶

AI systems are easy to improve by feeling and hard to improve by evidence.

A prompt can look better in a demo and still perform worse across real cases.

Evaluation answers:

Did the system actually get better?

Why normal unit testing is not enough¶

Traditional tests often look like:

assert add(2, 2) == 4

For an LLM response, many valid outputs may exist.

Question:

Summarize this support ticket in one sentence.

Several summaries could all be correct.

Therefore AI evaluation usually combines:

deterministic checks,
reference examples,
semantic checks,
model-based judging,
human review.

Start with a representative dataset¶

Create a set of real or realistic inputs that represent the task.

Example for support classification:

input                                      expected
-------------------------------------------------------------
"I was charged twice"                    billing
"The app crashes when I sign in"         technical
"Please change my email address"         account

A useful dataset contains easy cases and edge cases.

Avoid evaluating only on the examples used while writing the prompt.

Golden examples¶

A golden dataset contains examples with known expected behavior.

For extraction:

{
  "input": "Invoice INV-42 total 120 EUR",
  "expected": {
    "invoice_number": "INV-42",
    "currency": "EUR",
    "total": 120
  }
}

For classification, exact expected labels work well.

For open-ended generation, expected properties may be more useful than one exact answer.

Deterministic evaluation¶

Use deterministic checks whenever the requirement itself is deterministic.

Examples:

Schema correctness¶

Is the output valid against the required schema?

Classification accuracy¶

predicted category == expected category

Required citations¶

Does every factual answer contain at least one source reference?

Safety rules¶

Did the system invoke a restricted tool without authorization?

These checks are cheap, repeatable, and easy to understand.

Exact match¶

Exact match is useful for tasks such as:

IDs,
labels,
boolean decisions,
normalized values,
structured extraction.

Example:

expected: "billing"
actual:   "billing"

For natural-language generation it is usually too strict.

Two correct answers can use completely different wording.

Property-based evaluation¶

Instead of comparing the full answer, test required properties.

For a generated email:

- must be under 150 words
- must mention order ID
- must not promise a refund
- must use a professional tone

Some properties can be checked deterministically, others need semantic evaluation.

Semantic evaluation¶

Semantic checks evaluate meaning rather than exact text.

Examples:

did the answer preserve the key facts?
did the summary omit an important issue?
did the response answer the user's question?
is the answer supported by the retrieved context?

This may involve embeddings, classifiers, an evaluator model, or human review.

LLM-as-a-judge¶

Another model can evaluate outputs using explicit criteria.

Example evaluator instruction:

Score the answer from 1 to 5 for factual consistency with the provided source.
Do not score writing style.
Return only the score and a short reason.

This can scale evaluation better than manual review, but the judge is also probabilistic.

Therefore:

define a clear rubric,
test the judge against human labels,
use deterministic checks where possible,
avoid pretending the judge is ground truth.

Rubrics¶

A rubric makes subjective criteria explicit.

Example RAG-answer rubric:

5 - fully answers the question and all factual claims are supported
4 - correct but misses a minor detail
3 - partially correct or weakly supported
2 - significant errors
1 - mostly incorrect or unsupported

Without a rubric, evaluator scores are harder to compare over time.

Human evaluation¶

Humans remain useful when:

quality is highly subjective,
domain expertise is required,
safety consequences are important,
automatic evaluation is not trustworthy enough.

Human review can also create the labeled dataset needed to automate future evaluations.

Evaluate components separately¶

An AI application may fail because of different components.

Example RAG system:

query
 ↓
retrieval
 ↓
context
 ↓
generation

A bad answer can mean:

retrieval found the wrong documents,
relevant documents were ranked too low,
context was truncated,
the model ignored correct context,
the model hallucinated.

If you evaluate only the final answer, diagnosing the failure is harder.

Retrieval evaluation¶

Common questions:

Did we retrieve the relevant document?
Was it in the top K?
How much irrelevant context did we retrieve?

Possible metrics include:

precision,
recall,
hit rate,
ranking metrics.

The exact metric matters less initially than separating retrieval quality from answer quality.

Tool-use evaluation¶

For tool-enabled agents evaluate:

did it choose the correct tool?
were arguments correct?
did it call unnecessary tools?
did it stop at the correct time?
did it violate permission boundaries?
did it recover correctly from tool failure?

Example test case:

User: "What is the status of order 123?"
Expected tool: get_order
Forbidden tool: cancel_order

Trajectory evaluation¶

For multi-step agents, the final answer may be correct even if the path was inefficient or dangerous.

Trajectory:

search
 ↓
read file
 ↓
search again
 ↓
run test
 ↓
edit
 ↓
run test

Evaluate not only the final result but also:

number of steps,
unnecessary calls,
repeated calls,
risky actions,
cost,
recovery behavior.

This becomes important for agentic loops later.

Regression testing¶

Every prompt, model, retrieval, or tool change can alter behavior.

Treat AI changes like software changes:

change
 ↓
run evaluation suite
 ↓
compare against baseline

Example:

Version A
accuracy: 87%
avg latency: 1.8 s
avg cost: $0.012

Version B
accuracy: 91%
avg latency: 3.6 s
avg cost: $0.031

Version B is more accurate, but the trade-off may or may not be worth it.

Baselines¶

Always compare against something.

Possible baselines:

previous prompt,
previous model,
simple keyword classifier,
deterministic rule system,
human performance,
no-RAG model answer.

Without a baseline, a metric has little context.

Offline versus online evaluation¶

Offline¶

Run known test cases before release.

Good for:

regression tests,
model comparisons,
prompt experiments.

Online¶

Measure real production behavior.

Examples:

user satisfaction,
escalation rate,
tool failure rate,
task completion rate,
correction rate,
latency and cost.

Offline evaluation protects releases. Online evaluation shows whether the product works in reality.

Example: classification change¶

Suppose you change the support-classification prompt.

Weak process:

try 5 examples manually
 ↓
looks better
 ↓
deploy

Better process:

200 labeled historical tickets
 ↓
run old prompt
 ↓
run new prompt
 ↓
compare accuracy by category
 ↓
inspect regressions
 ↓
measure latency and cost
 ↓
decide whether to deploy

Example: coding agent¶

Evaluation dataset:

20 small repository tasks

Metrics:

- tests passing
- compilation success
- task completion
- number of files unnecessarily modified
- tool calls
- total tokens
- elapsed time

This is much more useful than asking whether generated code "looks good".

Avoid one-number evaluation¶

AI systems often optimize multiple dimensions:

quality
latency
cost
safety
user experience

A model that gains 1% quality while increasing cost 10x may be a worse product decision.

Use a small scorecard instead of one magical metric.

Build evaluations early¶

A common mistake is:

build entire AI product
 ↓
then think about evaluation

Better:

define task
 ↓
collect representative examples
 ↓
define success criteria
 ↓
build
 ↓
evaluate continuously

Evaluations become the feedback loop for AI engineering.

Practical first evaluation setup¶

For a new AI feature:

collect 20–100 representative cases,
define expected outputs or properties,
add deterministic checks first,
add an evaluator model only where necessary,
save baseline metrics,
run the suite after prompt/model changes,
inspect failures manually,
grow the dataset from production failures.

Mental model¶

AI development without evals
        =
manual guessing at scale

A better loop is:

hypothesis
 ↓
change
 ↓
evaluation
 ↓
error analysis
 ↓
next change

The purpose of evaluation is not to prove the model is perfect. It is to make AI development measurable, comparable, and less dependent on anecdotes.