A new arXiv paper introduces WebRider, a benchmark for live-web assistance that evaluates more than whether an agent reaches a final answer. The authors argue that delegating a web task also means transferring a policy: what to verify, which preferences matter, how to handle uncertainty, and when to stop.
That framing addresses a weakness in many web-agent evaluations. A model can produce a plausible final result while violating the user’s constraints along the way, such as skipping checks or making choices the user did not authorize. WebRider uses persona-conditioned intent controllers to examine whether agents follow those constraints during live web tasks.
The work is research, but the practical issue is immediate. As agents book services, compare products, and operate websites on a user’s behalf, success cannot be judged only by the final output. Systems need to preserve user intent throughout the process, especially when the web page changes or hidden tradeoffs appear.