A Hugging Face experiment shows that training a search model for compression can preserve more retrieval quality than compressing its vectors only after training. The method targets multi-vector embedding systems, which keep a representation for many document tokens and can consume substantial storage at large scale.
Token pooling merges similar document vectors. With a pooling factor of four, a conventionally trained model retained 94.6% of its uncompressed NanoBEIR nDCG@10 score. Pooling-aware training raised retention to 97.8% at the same fourfold reduction in vector count. In the example calculation, compressing 200 vectors per document to 50 cuts residual storage for one million documents from 12.8 GB to 3.2 GB at the stated precision.
The technique uses hierarchical clustering to merge nearby token representations and trains the encoder to keep useful information after those merges. Results come from a specific model, benchmark and compression setup, so production teams should measure recall and latency on their own corpus. The accompanying Sentence Transformers example makes that trade-off reproducible rather than presenting compression as universally lossless.