Most AI pilots do not fail — they simply never end. A demo is run, it looks good on ten examples, someone asks "but what about the complicated cases", and there is no answer because nobody defined what is being tested. This guide is a business test plan: how to define a pilot so that at the end there is a table with numbers, and a decision follows from the table. It does not explain what a language model is or how to build RAG — it explains what the process owner should demand before anyone starts building.
Define the action and the user — and the existing alternative
A pilot tests one action performed by one user, and compares it with how it is done today. The three parts must be written in a single sentence before moving on:
| Part | Question | Example (illustration) |
|---|---|---|
| The action | What exactly will the system do? Answer from knowledge, help with phrasing, process a document, or make a defined decision? | "Answer service reps' questions about return policies, from the policy documents" |
| The user | Who performs the action, in what context, and how often? Can they judge an answer? | "A service rep on a call with a customer, several times an hour; knows the domain and will spot a wrong answer" |
| The existing alternative | How does it happen today, how long does it take, and what is the common error? | "The rep searches a PDF or asks a colleague; takes a few minutes; the common error — an outdated policy" |
Before moving on to a pilot, one question: can the existing alternative be improved without AI? Better document search, an updated policy document or a structured form sometimes solve the same problem cheaply and predictably. The four questions that decide whether AI fits the need — required quality, data, cost and the alternative — are set out on the service page. AI in your product — is AI right for the need, and what the pilot includes
What counts as success? Quality, time, adoption and cost
Success conditions are set before the pilot, in writing, with the process owner — the person responsible for the business outcome, not for the technology. Four measures, each with a target the process owner fills in:
| Measure | How it is measured | Target (filled in by the process owner) |
|---|---|---|
| Quality | The share of answers in the scenario set a qualified person would approve as correct and complete; separately — the share of wrong answers given with confidence | "At least ___ of ___ scenarios approved; no more than ___ confidently wrong answers" |
| Time | Time per action with the system versus the existing alternative, including the time to check the answer | "At least ___ shorter than the alternative, including checking" |
| Adoption | How many users in the trial group keep using it after the first week, without being asked | "___ of ___ users active in week three" |
| Cost | Usage cost (model calls, retrieval, storage) per action, and a forecast at the monthly action volume | "No more than ___ per action; no more than ___ per month at full volume" |
Test data and boundaries
The pilot is tested on a scenario set — a collection of real or representative cases, with the answer or outcome a person would accept. The set is built before the system is, by people who know the domain rather than those building it, and deliberately contains what demos tend to leave out:
- The regular cases — most of the set, distributed like real use.
- Edge cases — unclear phrasing, a question touching two policies, conflicting information in the sources.
- Questions the sources cannot answer — the right answer is "I don't know" or a referral to a person, and that is tested like any other scenario.
- Attempts to get out of the system what is not allowed — another customer's information, an action without permission.
- Part of the set is held back and not shown to the builders until the final check — otherwise the system "learns" the exam.
Alongside the set, define boundaries: which sources the system may access, and under which permission — the same permission as the user it serves, no more; which data may be sent to the model provider and which may not (and what its contractual terms are on data use); what is kept in logs and for how long. If the data includes personal information, these definitions are closed with whoever is responsible for privacy in the organisation before the pilot, not after.
What happens when the system fails? Uncertainty, escalation and action approval
An AI system will fail in some cases — not a question of whether but of how. The difference between a pilot that can be approved and one that is dangerous to continue is what the system does at the moment of failure. Three mechanisms, each tested in the scenario set:
| Mechanism | What the system does | What is tested |
|---|---|---|
| Uncertainty | When there is no answer in the sources or they conflict — says so, shows what it found, and does not fill in from imagination | Every unanswerable question in the set: how many got "I don't know" and how many got an invented answer |
| Escalation to a person | In defined cases — a sensitive topic, an angry customer, a question outside scope — hands over to a person with a summary of what was already said | The scenarios defined for escalation were in fact escalated; the handed-over summary is correct and complete |
| Action approval | Any action that changes data, sends a customer a message or charges money — shown to a person for approval before execution, at least in the pilot | No sensitive action was executed without approval; the proposed actions were correct |
Experiment sheet — a template with acceptance criteria
Everything above converges into one sheet, signed by the process owner before starting. This is the template; the examples are filled in for illustration only.
| Section | Content (example) |
|---|---|
| Action · user · alternative | Answering return-policy questions · a service rep on a call · searching a PDF or asking a colleague |
| Permitted sources | The current version of the policy documents only; no access to customer data |
| Trial group and duration | ___ reps from one shift, ___ weeks |
| Scenario set | ___ documented real questions (approved for use) + ___ edge cases + ___ questions with no answer in the sources; ___ of them held back for the final check |
| Acceptance criteria | Quality: ≥ ___ approved, ≤ ___ confidently wrong · Time: ___ shorter than the alternative · Adoption: ___ of ___ in week 3 · Cost: ≤ ___ per action |
| Example input | "Customer bought 40 days ago and wants to return, has a receipt" |
| Expected output | A quote of the relevant clause, an answer on whether the return is possible per policy, and a pointer to the exceptions clause — with a link to the document |
| Example failure (tested) | "What is the policy for products bought on the partner's site?" — no policy exists. Correct answer: "I have no information on this in the policies, hand to the shift manager". Failing answer: inferring from another policy. |
| Actions requiring approval | None — in the pilot the system only answers, it does not act |
| Who judges, who decides | Judge: the service team lead on every scenario · Decider: the head of service (process owner), at a fixed date at the end of the pilot |
The decision after the pilot: continue, fix, narrow or stop
The pilot ends on the date that was set, not when the results are good. On that date the process owner makes one of four decisions, according to the resulting table:
| Decision | When | What the next step is |
|---|---|---|
| Continue | All four criteria passed, and the failure mechanisms worked in every scenario meant for them | Define what production requires — permissions for all users, monitoring, a usage budget, an update process and support — as a separate project with its own scope |
| Fix | Quality is close to the target, and failures cluster in an identifiable pattern (a missing source, one question type) | One short, defined round — what is fixed, and a re-test on the held-back part of the set. Not "a bit more" |
| Narrow | The system is good on some scenarios and poor on others, and the split between them is clear | Limit the scope to what works (policies only, not pricing, for example), and escalate the rest to a person by default |
| Stop | Quality is far from the target, or cost per action exceeds the alternative, or users did not return — and there is no fixable pattern | Document what was tested and found, so the next question starts here and not from zero. This is a legitimate pilot outcome — it is why you run one |
What we do at this stage
Before AI goes into a product we define the experiment sheet with you: action, user and alternative; success conditions with the process owner; a scenario set and boundaries; and failure behaviour. Then we build the pilot at the defined scope, run it with the trial group, and deliver the table — scenarios, score, cost — on which the decision is made. What the pilot includes, how quality and failures are tested, and what it takes to turn it into a stable product — on the service page, including exactly what we built in the published cases. AI in your product, with a clear test process
When the AI is part of a new product that has not launched yet — the pilot is part of defining the first version, and the decision after it determines whether the capability goes into the MVP or is deferred. When it is added to an existing business system — it enters the same anatomy of a system: permissions, data, integrations and monitoring. MVP development — how a first version is defined and built
What a business system includes beyond its screens — and where an AI capability fits in it — in the anatomy section of the software development page. Custom software development — the anatomy of a system