Woodpecker Distillation uses weak models to patch strong-model reasoning

Woodpecker Distillation uses weak models to diagnose and repair local reasoning bugs in stronger language models.

The paper argues that many failures on reasoning tasks are not proof of global incompetence. Instead, a model may take a wrong turn at an intermediate step. The authors show that inserting a short patch from a weaker probe model after the strong model’s own reasoning prefix can redirect the answer toward a correct solution.

Simply fine-tuning on those patches or repaired trajectories did not reliably internalize the improvement. The proposed method instead contrasts successful and unsuccessful weak-model patches at the same prefix, builds a corrective teacher distribution from the future predictions they induce, and distills that signal into the strong model.

The approach is important because it treats reasoning errors as local and repairable. If it generalizes beyond the tested math benchmarks, weak models could help train stronger systems by identifying where their reasoning trajectory starts to fail.