AWS has described the quality-control pipeline behind NarrateAI, an internal assistant used by more than 4,000 company executives for live business questions. The system combines five techniques intended to keep answers fast while preventing a language model from inventing metrics. AWS reports approximately 99 percent numerical accuracy, though the post presents an internal implementation rather than an independent benchmark.
The first layer routes queries according to the amount of retrieved data. Roughly 90 percent fit in a single model call and take a fast path; larger questions are divided into parallel batches and consolidated afterward. Models can also fail over across AWS accounts, creating separate quota pools when one endpoint is throttled.
Quality checks begin before generation finishes. NarrateAI evaluates each paragraph as it streams, overlapping verification with the creation of the next section. Multiple evaluators check different qualities in parallel, and a numerical cascade first looks for exact values in the retrieved records before escalating ambiguous cases to semantic review. That preserves a cheap, deterministic check for straightforward figures.
The architecture illustrates that production accuracy comes from system design rather than one model choice. Routing, capacity management, evaluation and direct comparison with source data each address a different failure mode. Teams adopting the pattern would still need to calibrate thresholds on their own documents and measure errors on held-out questions relevant to their decisions.