Retry costs can make cheap AI models more expensive

A new arXiv paper argues that AI agent costs are often mispriced because they ignore retries. When a model fails on the first attempt, an agentic workflow may call it again, use more context, or branch into extra reasoning, turning a cheap per-token model into an expensive end-to-end choice.

The authors call this gap token inflation: the ratio between a workflow's real cost and the cost implied by a single model call. They report inflation as high as 4.25 times for a 7B model on multi-hop question answering and say existing routers can underestimate true cost by more than two times on difficult tasks.

Their proposed InflationAgent router measures this overhead and uses pre-execution signals to choose models more realistically. The practical lesson is broader than the specific method. Buyers and developers should compare models by completed task cost, not just sticker-price tokens, especially when agents are allowed to retry until they get a usable answer.