The checklist we run before an AI feature ships

Normal software either works or throws an error. AI features do something stranger: they work most of the time, and when they don't, they're confidently wrong in a full sentence.

After one launch where an assistant cheerfully told a user their (non-existent) meeting had been moved, we wrote a checklist. It's not a framework. It's a list in a doc, and the feature doesn't go out until each line is ticked.

Before launch

1. We have at least 30 real examples. Real inputs, written by real users or pulled from logs, with what a good answer looks like. Not inputs we invented at our desks. Ours always turn out too tidy.

2. We ran them and read every output. A score is useful later. The first time, a person reads all of them. You find problems no metric would have caught, like the bot replying in English to a Tamil question.

3. We included the awkward cases. Empty input. Very long input. A question the feature shouldn't answer. Someone trying to get it to ignore its instructions. A request that needs approval.

4. We know what "wrong" costs. A bad summary is annoying. A wrong payment is a phone call from a lawyer. The riskier the action, the more of it goes through a human approval step, in code.

5. There's a fallback. If the model times out or refuses, what does the user see? "Something went wrong" is not a fallback.

6. We can see what it's doing in production. Inputs, outputs, tool calls, latency and cost per request, stored somewhere we can search. Without this you find out about problems from angry emails.

7. We know the cost per task. Multiply it by expected usage. Then multiply that by three, because usage always grows faster than you planned.

After launch

8. Wrong answers become test cases. Every bad answer a user reports goes into the example set. The set grows. The next change is tested against all of it.

9. Re-run when anything changes. New model, new prompt, new tool, new provider. Re-run the set and compare before switching. This is the line that lets us change models in an afternoon instead of a sprint.

That's it

Nine lines. Nothing about which eval library to use, because it matters much less than actually having the examples and reading the outputs. A spreadsheet works. We started with one.

If you're about to launch something and want someone to try to break it first, email naveen@estreetz.com. We enjoy that part more than we should.

Written by Naveen at eStreetz. Questions or disagreements: naveen@estreetz.com