When a language model recalls a fact, it appears to establish the type of relationship before committing to the specific entity it will name. Researchers found this ordering across four decoder-only models and eight families of prompts using several interventions designed to identify which internal information actually controls the next token.
For a question about a national capital, the model may represent both the “capital of” relation and the country early in processing. The experiments found that relation information became causally active at the final token 10 to 16 tested layers before entity information, equal to 31% to 44% of network depth at the study’s main threshold.
The entity itself was not missing from early layers. Patching representations at the entity-token position succeeded in 90% to 100% of cases. Its influence on generation was deferred until that information had been routed to the final position where the answer is produced. The relation-first ordering held across all 16 tested combinations of models and thresholds from 0.2 to 0.5.
The work examines controlled factual-recall prompts rather than every form of reasoning or hallucination. Still, separating availability from causal control gives interpretability researchers a more precise account of recall: a model can contain the relevant entity representation well before it uses that representation to choose its output.