A new preprint proposes compressing the memory used to serve long-context language models while keeping every active request page addressable. Minima-KV targets the key-value cache, which stores earlier attention states and can become a major capacity and bandwidth bottleneck during generation.
The system leaves recent pages and selected anchors in FP8 precision, then moves older, unprotected pages into a packed TQ3 format. Separate kernels process each format and merge their attention results with shared normalization. That avoids building a full dense copy of the compressed cache before every decoding step.
The authors evaluated configuration-specific Qwen3.6-27B setups on a single 96 GB Nvidia RTX Pro 6000 Blackwell GPU. They report that the hierarchy can retain live context while allowing more concurrent work than an all-FP8 cache. The paper also examines the quality and throughput trade-offs introduced by moving older states to lower precision.
These are preprint results from one hardware family and selected workloads, not a universal serving guarantee. Operators would need to test whether the chosen anchors preserve quality for their prompts and whether format conversion and custom kernels justify the additional system complexity.