LinearKV tackles a serving problem created by newer hybrid language models that mix attention with linear recurrence layers.
Position-independent caching can speed up LLM serving by reusing chunks of previous computation. Existing methods assume full-attention models, where token-indexed key-value caches can be matched, concatenated, and locally repaired. Hybrid models break that assumption because most layers expose only a fixed-size recurrent state rather than a token-by-token cache.
The paper’s approach is a training-free framework that decouples initialization. Full-attention layers reuse ordinary KV concatenation, while each linear layer maps matched local states to a single initial state. The authors report that one cached state is enough as the initializer for the linear layer.
The work is technical, but the practical issue is straightforward: inference systems need caching methods that match model architecture. As hybrid LLMs become more common, serving optimizations built for classic transformers may need redesign rather than minor tuning.