Researchers propose treating the diagnosis of failed AI-agent runs as an iterative search problem rather than a single judgment over an execution log. Their Continual Search method asks a model judge over successive turns to keep looking for unresolved evidence before settling on a root cause.
Long-running agents can generate huge traces in which the decisive mistake is sparse, distant from the visible failure or spread across several actions. The authors argue that one-shot evaluators often seize on an early plausible explanation and leave relevant parts of a long record unexamined.
They tested Continual Search across four existing root-cause attribution benchmarks and introduced MegaRCA-Mix for larger traces. The new set contains 50 human-annotated failures from long-horizon, execution-heavy tasks. On MegaRCA-Mix, the method raised GPT-5.5’s F1 score from 0.349 to 0.498, an improvement of more than 40% relative to the starting score.
The researchers also report cases where a lower-tier model using better search surpassed a higher-tier model in the same family. That suggests diagnostic procedure can matter more than scale alone. The work remains a new arXiv preprint, and a 50-trial dataset is limited, but it provides a concrete approach to turning unwieldy logs into actionable reliability findings.