Skip to main content

LLM Evaluation

Updated 2 min read

Share this page

Send the link, quote the definition with a link back, or show it as a card on your own site.

https://softwaredictionary.org/terms/llm-evaluation

In short

LLM evaluation, or evals, is measuring how well a language model or an AI feature performs on a fixed set of test cases, so that changes can be compared.

What is LLM evaluation (evals)?

LLM evaluation, usually called evals, means testing an AI model or a feature built on one against a fixed set of examples. Each example has an input and either an expected answer or the criteria a good answer must meet. You run the model on all of them, score the results and compare the score between versions: a new prompt, a different model or a change to the retrieval step.

There are two main kinds. Public benchmarks, such as MMLU for general knowledge or SWE-bench for fixing real code, compare models with each other. Your own evals, built from the questions and documents your product really sees, tell you whether the feature works for your users. Answers are scored in four common ways: an exact check, code that verifies the output (for example that it is valid JSON or that tests pass), another language model acting as a judge with a rubric, and human review.

Evals are to AI features what a test suite is to code. A few spot checks in a chat window can't tell you whether a prompt change helped, just as trying a function once doesn't prove it works; a fixed set of cases, run on every change, can.

A common misconception is that a high benchmark score means a model will do well on your task. Benchmarks measure general skills, their questions can leak into training data, and they don't contain your users' questions. Even 50 to 100 well-chosen cases from your own product, kept in version control and run in CI, say more about your feature than any leaderboard.

Key takeaways

  • Evals measure an AI model or feature on a fixed set of test cases.
  • Public benchmarks compare models; your own evals measure your feature.
  • Answers are scored by exact checks, code, an LLM judge or people.
  • Run evals on every prompt or model change, like tests in CI.
  • A high benchmark score doesn't guarantee good results on your task.

Example

A tiny eval: score a support bot before and after a prompt changepython
cases = [
    {"input": "Can I get a refund for order 1234?", "must_include": "refund"},
    {"input": "How do I cancel my subscription?", "must_include": "cancel"},
    {"input": "Do you ship to Canada?", "must_include": "canada"},
]

passed = 0
for case in cases:
    answer = support_bot(case["input"]).lower()
    if case["must_include"] in answer:
        passed += 1

print(f"{passed}/{len(cases)} passed")  # compare with the score before the change

Readers ask

What is LLM-as-a-judge?

Using one language model to grade another model's answers against a rubric. It scales far better than human review, but judges have biases, such as favoring longer answers or the first answer they see, so check them against human ratings.

How many test cases do I need?

Start small: a few dozen well-chosen cases covering common and tricky inputs catch most regressions. Add a case every time you find a failure in real use, so the same mistake can't come back unnoticed.

What is the difference between evals and benchmarks?

A benchmark is a public, fixed eval used to compare models. Evals is the broader practice, including the private test sets a team builds to check its own product.

Free · once a week

Software and AI, explained once a week

The week's software and AI news with every term explained, plus a term to learn and a quick quiz. A two-minute read.

You confirm by email first, and one click unsubscribes. Privacy

See this week's issue

About this page

Spotted a mistake or something missing on this page?Suggest an edit

How pages are made

More

Settings