Researchers have tested token merging as a way to make multilingual speech recognition less expensive to run. The technique combines redundant intermediate features, shortening the sequence a model processes during inference without requiring the model to be retrained.
The study applied token merging to three sizes in the Whisper model family and evaluated them across 16 languages. Its authors report increased computational efficiency with almost no reduction in transcription accuracy across most of the low-resource languages and model sizes they examined.
They also tested the method after adapting Whisper to low-resource languages with DoRA, a parameter-efficient fine-tuning technique. Token merging continued to work after that customization, suggesting teams would not necessarily have to choose between language-specific tuning and faster inference.
The result is relevant to deployments where compute and latency make large multilingual models difficult to use, especially for languages that lack dedicated systems. The paper is a new arXiv preprint, so its findings have not yet passed peer review. It nevertheless provides a systematic comparison across model scales and languages rather than demonstrating the optimization on a single benchmark.