AWS has published experiments on Self-Distilled Reasoning, a technique for fine-tuning Amazon Nova models when training data lacks high-quality chain-of-thought traces. The method reuses reasoning generated by the base Amazon Nova 2 Lite model as a stand-in for validated teacher traces.
Supervised fine-tuning can improve a model on a target dataset, but it can also cause catastrophic forgetting, where earlier capabilities deteriorate. AWS reports that vanilla fine-tuning caused average math performance to fall from 70 percent to 6 percent in one set of experiments.
With Self-Distilled Reasoning, AWS says the model retained much more of its general reasoning ability while matching or improving target-task performance. The post compares the approach with model merging, another method used to recover lost skills after fine-tuning.
The work is a technical exploration rather than a guaranteed recipe for every dataset. Its useful takeaway is narrower: if a customization dataset has answers but no reasoning traces, letting the base model generate structured reasoning may regularize training and reduce unwanted skill loss.