The problem
Judge an assistant by the answers it should not give
2,000 articles, 30,000 questions a month: answer with citations or hand off, in under 5 s.
A software company has two thousand help articles and thirty thousand customer questions a month. It wants an assistant. The assistant should answer from the articles and show which ones it used, or hand the question to a person when it is unsure, in under 5 seconds, within budget. A demo takes an afternoon. Knowing whether it is good enough is the work.
The estimate
Cost follows tokens, and most tokens are the articles
2,660 tokens in, 250 out. About $29 a month on a small model, $352 on a large one.
The question is about sixty tokens. The prompt around it, with five chunks of articles, is about 2,660, and the answer about 250. At example prices, a small model costs about $29 a month for thirty thousand questions, and a large one about $352. Both are small beside a person answering each one. The real question is whether each step is worth its cost and delay.
The first design
Asked directly, a model answers from training, not from your product
The question goes straight to the model. It answers fluently, and invents a feature and a price.
The first version sends the question straight to a model with a short system prompt. Cheap and fast, and the demo impresses everyone. Asked whether reports can be exported to Excel, it says yes, free on every plan. In fact export is CSV only, and only on the Business plan. The model has never seen this help centre.
The cause
Measured on 200 real questions, 42% of answers are wrong
An evaluation set of 200 past questions: 58% correct; the rest made up, out of date, wrong product or unsafe.
We take 200 real past questions, with answers written by support staff, and grade every answer. Only 58 percent are correct. The rest fall into groups: made up, out of date, about the wrong product, and a few that were unsafe. Each group points to a different fix.
The fix
Retrieval gives the facts; a guardrail and a hand-off stop the rest
Search, rerank, answer only from sources, check, or hand off. Wrong answers fall from 84 to 10 of 200.
The articles are chunked and indexed for keyword and vector search. For each question, the best matches are reranked, the model answers only from them and cites each one, a guardrail checks the answer is grounded and safe, and if search finds nothing, a person takes it. Before: 116 of 200 correct. After: 172 correct, 18 handed to a person, and 10 wrong.
The trade-off
Every step that improves answers adds cost and delay
The small model with a reranker and guardrail: 0.28¢ an answer, 3.9 s. A large model breaks the 5 s limit.
A small model with retrieval: well under a cent an answer, 2.4 seconds, 81 percent correct. Add a reranker and a guardrail: 86 percent, still under a cent, 3.9 seconds. A large model gains two more points, costs about 5 times as much, and takes 6.2 seconds, over the limit. Good retrieval did more for quality than a bigger model.
The finished design
The model is one box of eleven
An index pipeline, a query pipeline, a guardrail, evaluation and monitoring. Each box links to its lesson.
Here is the whole design. Each night, changed articles are chunked, embedded and indexed. For each question: search and rerank, a prompt with the sources asking for cited JSON, a small model, a guardrail, then send or hand off. The evaluation set runs on every change and blocks one that scores worse, and monitoring watches cost, latency and hand-offs. Each box opens the lesson that teaches it.
A real case
At DoorDash, the guardrail and the judge did most of the work
DoorDash’s driver assistant: RAG, a two-tier guardrail and an LLM judge. Hallucinations down 90%.
DoorDash built an assistant for its delivery drivers. It finds similar solved cases and puts their help articles into the prompt. Every answer must pass a guardrail before a driver sees it: a cheap similarity check first, then a language model reviews anything flagged. A judge model scores quality over time, and tests run on every prompt change. DoorDash reports ninety percent fewer hallucinations and ninety-nine percent fewer potentially severe compliance issues. The same shape as ours.
Lesson complete
You finished “A support assistant over company documents”
Nicely done.