LLM Evaluation Scorecard: A Practical Framework for Testing Prompts and AI Apps

Overview

Prompt testing is easy to underestimate. A revised system prompt may improve one example while weakening another. A model change may reduce latency but produce less complete answers. A retrieval update may increase factual coverage while introducing irrelevant context. Without a shared evaluation process, these trade-offs are usually discovered through scattered manual checks or user complaints.

An effective LLM evaluation framework turns those observations into a repeatable workflow. It defines what the application should do, which test cases represent real usage, how outputs will be scored, and what level of change requires review. The goal is not to assign a perfect objective score to every response. The goal is to make decisions more consistent and expose meaningful differences between prompts, models, datasets, and application versions.

Use the scorecard for chat assistants, summarizers, extraction pipelines, RAG systems, classification workflows, and tool-using applications. It can support early prompt experiments as well as release checks in a larger AI development workflow.

Evaluation should complement observability rather than replace it. For a related implementation checklist, see what to log for prompts, responses, costs, and failures.

Template structure

Store each evaluation as a versioned record so that results remain understandable after the prompt or model changes. A practical scorecard contains five parts.

1. Test case metadata

• Case ID: A stable identifier such as support_014 or rag_policy_006.
• Task type: For example, summarization, extraction, question answering, classification, or tool selection.
• Input: The user request, document, conversation, or structured payload used in the test.
• Expected behavior: A short description of what a good response must accomplish.
• Risk level: Low, medium, or high, based on the consequence of an incorrect result.
• Source version: The dataset, document snapshot, or fixture version used by the test.

2. System configuration

Record the variables that can affect the output. Include the system prompt version, user prompt template, model identifier, relevant settings, retrieval configuration, available tools, and application build. For a RAG workflow, also record the retrieved context or document IDs. This prevents a test result from being attributed to prompt engineering when the real cause was a changed chunking or retrieval process.

3. Quality criteria

Separate criteria instead of relying on one general impression. A useful baseline includes:

• Correctness: Does the response agree with the available facts or reference answer?
• Completeness: Does it cover the required points without omitting important information?
• Instruction following: Does it follow the requested format, audience, scope, and constraints?
• Grounding: When context is supplied, does it use that context accurately and avoid unsupported claims?
• Safety and policy fit: Does it handle prohibited, sensitive, or adversarial inputs as intended?
• Format validity: Is the result parseable and usable by downstream code?
• Efficiency: Is latency, token usage, or tool activity within the application's acceptable range?

Not every criterion applies to every task. A JSON extraction workflow needs format validity and field-level accuracy. A writing assistant may need relevance, tone, and completeness. A tool-using agent needs correct tool selection and parameter construction.

4. Scoring method

Choose a scale that reviewers can apply consistently. A simple four-point scale works well:

• 0 — Fail: The output is unusable, unsafe, or materially wrong.
• 1 — Weak: It shows partial progress but requires substantial correction.
• 2 — Acceptable: It meets the minimum requirement with minor issues.
• 3 — Strong: It meets the requirement clearly and needs little or no correction.

Define a release threshold before reviewing results. For example, a workflow may require no failures on high-risk cases, a minimum average for correctness, and valid structured output on every machine-checked test. Keep hard gates separate from averages: a high overall score should not hide one critical failure.

5. Evidence and decision

Save the actual output, reviewer notes, automated checks, and final decision. Label the result as pass, investigate, or fail. Include a short explanation for each non-passing case. These notes become valuable regression documentation when the team revisits the prompt months later.

How to customize

Begin with real tasks rather than artificial questions. Collect representative inputs from support tickets, product workflows, internal documents, or anonymized production traces. Include ordinary cases, ambiguous requests, empty or malformed inputs, long inputs, multilingual inputs when relevant, and known failure cases. A small, diverse test set is often more useful than a large collection of nearly identical examples.

Divide the set into three groups:

1. Smoke tests: A small group run on every change to catch obvious breakage.
2. Regression tests: Previously failed or high-value cases that must continue to work.
3. Exploratory cases: New or unusual inputs used to discover behavior outside the fixed test set.

Use deterministic checks wherever possible. JSON parsing, required-field validation, enum checks, regular-expression checks, citation presence, and exact classification labels can be automated. For structured responses, pair the scorecard with JSON schemas, function calling, and parsing reliability guidance. A response that sounds correct but cannot be parsed should not receive full credit in a software pipeline.

For subjective criteria, write a short rubric with observable descriptions. Instead of saying “good summary,” specify that the summary must identify the decision, include the stated deadline, exclude unsupported recommendations, and remain within the requested length. If human reviewers disagree often, the criterion is probably too vague or the examples need clarification.

LLM-as-judge evaluation can help review larger collections, but treat it as an assessment aid rather than unquestioned truth. Give the evaluator the rubric, relevant reference material, and a fixed output format. Periodically compare its decisions with human reviews, especially for high-risk or ambiguous cases. Pair judge-based scores with targeted human review and deterministic tests.

When comparing two prompts or models, run both against the same inputs and configuration as far as practical. Record the comparison unit clearly: prompt version A versus prompt version B, model X versus model Y, or retrieval pipeline before versus after. Review both aggregate results and individual disagreements. The average score tells you whether the change helped overall; the disagreement list tells you where the behavior changed.

Examples

Example: customer support classification

The application assigns each message one category and returns a JSON object containing category, confidence, and reason. The scorecard can use field-level checks:

• Category matches the labeled reference: 0 or 3.
• Required keys are present and correctly typed: 0 or 3.
• Reason is consistent with the message: 0 to 3.
• Confidence is within the permitted range: 0 or 3.

A release gate might require valid JSON for every case and no category failures among high-priority examples. The reason field can receive a separate review score because it is more subjective than the label.

Example: RAG question answering

For a retrieval-augmented application, test the answer and the retrieval context separately. Check whether the expected source was retrieved, whether the answer is supported by that source, and whether the application declines to answer when the evidence is insufficient. Add adversarial cases containing instructions inside retrieved documents so that prompt injection handling is evaluated alongside answer quality. The prompt injection prevention checklist for RAG and tool-using apps can complement these tests.

Example: summarization

Use source documents with known facts and define required elements such as decisions, actions, owners, and dates. Score factual accuracy, coverage, unsupported additions, and format. Test both short and long documents, because a prompt that works on a short sample may not preserve important details when the context grows.

For every example, preserve the original input and output rather than only storing a numerical score. A reviewer should be able to understand why a result passed or failed without reconstructing the entire experiment.

When to update

Revisit the scorecard whenever the application, its inputs, or its risk profile changes. Run the smoke suite for every prompt, model, parser, retrieval, tool, or dependency change. Run the broader regression suite before a release or when a change affects a critical workflow.

Add a new regression case after a meaningful production failure, user-reported issue, safety finding, or parsing error. Do not simply patch the prompt and close the incident; preserve the input as a test so the same failure is less likely to return. Review the scoring rubric when reviewers repeatedly disagree or when a criterion no longer reflects the product requirement.

Update configuration metadata when model names, prompt versions, retrieval settings, tool definitions, or output schemas change. If costs or latency matter, record those measurements under the same test conditions and review them alongside quality. Guidance on reducing LLM application costs without sacrificing output quality can help frame those trade-offs.

To put this framework into practice, create a versioned file with 20 to 50 representative cases, define three to seven criteria per task, and mark the cases that are hard release gates. Run the current and proposed versions side by side, inspect every failure and disagreement, then record the decision and rationale. After deployment, monitor real failures and promote the most useful ones into the regression set. This turns prompt testing from a one-time review into an ongoing quality loop.