Q-CARE proposes query-agnostic checks for RAG answer quality

A new arXiv paper proposes Q-CARE, a framework for evaluating retrieval-augmented generation without relying on fixed reference answers. Retrieval-augmented generation, or RAG, improves model factuality by grounding responses in retrieved documents, but evaluating those systems is difficult when user queries vary widely.

The authors address that problem by decomposing queries into sub-queries and answers into atomic claims. Q-CARE then evaluates two linked questions: whether the retrieved evidence covers what the query needs, and whether the generated answer's claims are verifiable from that evidence.

The framework produces coverage-aware retriever metrics, including C-Prec@k and C-nDCG@k, along with generator metrics for completeness, conciseness, and verifiableness. On a human-annotated benchmark spanning eight datasets, the authors report higher correlation with human judgments than four existing RAG evaluation metrics, including RAGEval and RAGChecker.

The practical value is diagnostic. Instead of giving a single vague quality score, Q-CARE tries to separate retrieval failures from answer-generation failures. The paper also says code and data are publicly available, which should make independent testing easier.