Numan tests its AI health coach by risk category instead of one average score

Numan says the central challenge in building its Nu health coach was not automation but measuring whether conversations stayed useful without crossing into diagnosis, prescriptions, or clinical decisions. The UK digital-health company built the system with LangGraph and LangSmith and organized evaluations around distinct behavior and safety categories.

The team found that conventional single-turn tests were a poor fit for coaching. A fixed prompt and expected answer could not capture whether advice was appropriate in the context of a longer exchange, especially when a patient pushed against a boundary. A single average score also hid small numbers of serious failures because good routine responses outweighed them.

Numan instead developed simulated conversations, category-specific criteria, and gates for regulated boundaries. This lets the team inspect where the coach fails rather than treating overall quality as one number. The post is a company case study, not an independent clinical validation, and it does not show that the agent can replace medical care. Its useful lesson is narrower: high-stakes conversational systems need tests that preserve context and expose rare safety failures before deployment.