Testing AI is a different job

Robin Angland and Lydia Smith, two of our senior QA analysts, have passed the ISTQB Certified Tester AI Testing certification (CT-AI). It comes from the international body that sets the standards for software testing, and it is specifically about testing systems built on AI.

That last part is the point. Testing AI is not the same job as testing ordinary software, and treating it as if it were is where a lot of AI goes wrong.

Why AI breaks the usual rules of testing

Most testing rests on a simple idea: given this input, the system should produce that output, every time. You write the expected result down and check the software against it. It’s deterministic, so a test is a clean pass or fail.

AI does not play by those rules. A machine-learning model gives you a probability, not a certainty. The same input can produce different outputs. The behaviour depends on the data the model was trained on, and it drifts as that data changes. A large language model can be fluent, confident, and completely wrong. Often there is no single correct answer to check against at all.

So the questions change. Not “did it return the right value” but “is it accurate often enough, across the cases that matter, without bias, and does it fail safely when it isn’t sure”. Answering those takes different techniques: measuring performance with metrics like precision and recall, testing the data as carefully as the code, and, for generative AI, adversarial approaches like red teaming to find where a model can be pushed off the rails. That is the ground CT-AI covers, and its latest version now includes generative AI and retrieval-augmented systems specifically.

Why we put two people through it

We don’t lead with AI, and we don’t ship it just because we can. But when we do build with AI, we hold it to the same bar as everything else we make: it has to work, and it has to keep working once real people depend on it. QA is where much of that bar is enforced, and where a lot of AI risk lives.

Two senior analysts trained in how to test AI means the AI we build is tested by people who understand how it can fail, not just shown working once. It is also how we get better on purpose: real training, on work time, in the things our clients are going to need.

If you are building with AI and the question “how would we know it works” is nagging at you, that is the right question. It is one we help clients work through: see how we help with AI.