Skip to content

No. 2August 14, 20262 min read

The test comes before the build

We do not start building until we have run a model on the business's own calls or files and agreed what a pass looks like. It is the least glamorous week, and the one that decides everything.

A demo always works. That is what a demo is for. The question a business should ask is not whether the thing works in a demo, but whether it works on their own calls, their own documents and their own customers, who do not speak the way a demo does.

So the first thing we do is not build. It is test.

What the first week looks like

We take a sample of the real thing: a week of calls, a folder of the documents that come in, the questions the team gets asked most. We run a model on them, with no polish, and we look at the results together.

Some of what comes back is good. Some is not. A caller with a strong accent and a bad line. An invoice from a supplier whose layout changes every month. A question that depends on a rule nobody has written down. These are the cases that decide whether the system is worth building, and they only show up on real material.

Agreeing what a pass is

Out of that week comes one short page: what the system has to get right, on what sample, how often. It is agreed in writing before the build starts.

This does two things. It makes the build honest, because the system is tested against the same page before it goes live, and it goes live only when it passes. And it makes the decision to stop honest, because if it cannot pass, we keep working on it or both sides agree to stop. Either way nobody is paying for a system that sounds good and does not work.

What people learn in that week

Usually something about their own business. How many calls are about the same three things. How often a document arrives with a field missing. Which questions the team answers from memory because the written rule is out of date.

That knowledge is useful whether or not a system is built. It is the cheapest week of the whole thing, and we would not skip it even if a client asked us to.

More on how we work is on the Approach page, in writing, where it belongs.

Written by webridgeAI

We design, build and run AI systems inside businesses. The newsletter is what we learn doing that, in a few hundred words at a time.

The next edition, by email.

Sent when an edition is written, and nothing else.

You can unsubscribe at any time. Privacy

All editions