Agents trained with reinforcement learning can learn to invoke search or code tools because of irrelevant wording rather than because a task needs them. Controlled experiments found that these shortcut policies appeared after training cues became strongly associated with particular tools, then persisted when researchers changed the task so the tool was unnecessary.
The study combined factual questions and mathematical reasoning in synthetic environments. In counterfactual tests, spurious tool calls increased by as much as 39 percentage points. The behavior was not caused by dataset imbalance alone: it emerged only when an agent had already learned to use the target tool reliably. Cues that were semantically related to a tool strengthened the effect, suggesting that existing model associations and reinforcement rewards can combine into a misleading policy.
Researchers mitigated the problem with a dense decision-level reward. An LLM judge evaluated whether every proposed tool call was necessary, providing feedback at the action where the shortcut occurred rather than only scoring the final answer. This sharply reduced cue-driven calls without lowering task performance in the experiments. Synthetic tasks cannot capture every production workflow, and an LLM judge can make its own errors. Still, the result warns developers that endpoint accuracy may hide wasteful or risky tool behavior. Evaluations should vary irrelevant prompt features and inspect why each external action was selected.