Write the evaluation set before you choose a model

Most AI projects start with a model choice. Which provider, which size, which context window. It is the wrong first question, because there is not yet a way to tell whether the answer was right. The first artefact should be the evaluation set: a fixed collection of real inputs, each with the answer the business would accept, written before anything is built.

What goes into it

An evaluation set is small and specific. For a support assistant it is the questions customers actually ask, taken from closed tickets, with the answer a senior agent would send. For a document workflow it is real documents with the fields a person would extract. Each case records the input, the accepted output and a note on what would make an answer unacceptable, such as quoting a retired price or revealing another customer's data.

It needs the awkward cases as well as the common ones: questions the system must refuse, documents with missing fields, requests that look routine but need a human. Those are the cases that decide whether the assistant can go near a customer.

Who writes it

The people who own the work write it, with us. An engineer cannot decide what a good billing answer is; the support lead can. That conversation tends to surface the real problems early. Two senior people may give different answers to the same question, which means the source material disagrees, and no model will fix that. We record which source wins and why, and that register becomes part of the build.

What it changes

With the set in place, choices stop being matters of taste. A larger model, a different chunking strategy or a new prompt is scored against the same cases before it ships, and the change report shows which answers improved and which got worse. Cost and latency go in the same report, so a gain in quality is weighed against what it costs to run.

It also changes the release decision. Instead of a demo that looked convincing, there is a number of cases passed and a list of failures, each with an owner. The person who owns incorrect answers can decide whether that list is acceptable for an internal pilot, for drafts that an agent approves, or for nothing yet.

Keeping it alive

A set written once goes stale as the product and the questions change. We add a case every time a real answer is corrected, version the set alongside the code and rerun it on every change to prompts, sources or model. It stays small enough to read in one sitting, because a set nobody reads is soon a set nobody trusts.

None of this is complicated. It is the same discipline a team already applies to software tests, applied to a component whose behaviour cannot be read from its code. Starting there costs a few days and saves the much longer argument about whether the assistant is any good.