Adding a reranker after vector search raised precision at the first result from 81% to 97.1% in an OpenRouter experiment. The pipeline retrieved 30 candidate passages, scored each one against the question with Voyage Rerank 3 Lite, then supplied the best three to the answering model.
The test covered 274 questions linked to known passages in a synthetic documentation set. Reranking also raised final-answer accuracy to 99.3% while allowing the answering model to read three chunks instead of five. The rerank step added about $0.10 per 1,000 queries in that setup, making the complete pipeline roughly $0.04 more expensive after lower answer-generation costs.
The technique addresses a weakness of embeddings: passages about similar products or error codes can cluster together even when only one contains the exact answer. A cross-encoder compares the full question with each candidate, but it cannot recover a passage omitted by the initial search. It can also rank an unanswered discussion highly because relevance is not the same as correctness.
OpenRouter recommends starting with about 30 candidates and measuring precision, latency and total answer quality on production-derived cases. Its dataset was clean and model-generated, and prices and model behavior can change. Long passages can also be silently truncated by rerankers with small context limits.