SearchAuditor targets a growing problem in deep-search agents: long web interactions can fail because of a small reasoning error buried deep in a noisy trace.
The paper introduces SearchAuditBench, a benchmark of 1,243 failed trajectories collected from eight open-weight models on five deep-search benchmarks. The traces average 73.1 messages and 65.1 thousand tokens, and each is annotated with the critical error step, a root cause, a reference repair, and grading rubrics.
The authors also propose SearchAuditor, a multi-perspective auditing framework meant to localize, attribute, and repair failures through evidence-grounded adjudication. The goal is to reduce the human burden of manually inspecting long traces when an agent produces a fluent but wrong answer.
This matters because search agents are only as useful as their failure handling. If teams cannot identify where a long trajectory went wrong, they cannot improve the system reliably. Auditing may become a standard layer for agent evaluation.