Five Targeted Documents Cut Classical RAG Retrieval Scores by Up to 61.5%

A controlled benchmark of retrieval-augmented generation (RAG) systems found that five documents tailored to each query reduced classical retrievers’ top-10 ranking scores by 40.2% to 61.5%. The evaluation used 623 queries from SciFact and NFCorpus, two research benchmarks. In isolated-target tests, each query was paired with its own five targeted documents. The result does not describe five documents added once to a shared collection for all 623 queries. The study measured nDCG@10, a score for how well a system ranks relevant documents among its first 10 results. The reported decline applied to classical retrievers under the isolated-target setup.

Source: TokenPost