AWS proposes query-aware compression to lower RAG token costs

AWS has described a query-aware compression pattern for retrieval-augmented generation on Amazon Bedrock. Retrieval-augmented generation, or RAG, gives a model external context before it answers, but the retrieved text can make input tokens a significant cost.

The pattern uses a smaller model after retrieval to filter chunks against the user’s query before the primary model generates an answer. If the filter removes irrelevant context without dropping essential evidence, the final model has less text to process.

The approach is practical for teams trying to control AI costs, but it adds another decision point to the pipeline. Developers still need to test whether compression preserves answer quality for their documents, queries, and risk tolerance.