---
type: "post"
id: "rcp-ndcg-10-a-new-approach-to-enterprise-1059c0"
slug: "rcp-ndcg-10-a-new-approach-to-enterprise-1059c0"
title: "RCP-nDCG@10: A New Approach to Enterprise Retrieval Quality"
description: "Cohere’s thesis is that retrieval benchmarks can mis-score systems when their relevance labels omit useful documents."
retrieval_nugget: "Cohere’s thesis is that retrieval benchmarks can mis-score systems when their relevance labels omit useful documents."
published_at: "2026-09-30"
updated_at: "2026-09-30"
record_date: "2026-09-30"
date_kind: "published_at"
topics: []
source_urls: ["https://cohere.com/blog/rcp-ndcg"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:rcp-ndcg-10-a-new-approach-to-enterprise-1059c0"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:2af424ae68ee28bb26b3902833f62972bef21a5fed5f566be24954f97ccfd4b9","reviewed_at":"2026-09-30T16:03:10.640Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Cohere’s thesis is that retrieval benchmarks can mis-score systems when their relevance labels omit useful documents.","observed_facts":[{"text":"Cohere describes RCP-nDCG@10 as using a calibrated AI judge to assess retrieved documents against an explicit relevance rubric.","source_urls":["https://cohere.com/blog/rcp-ndcg"]},{"text":"Cohere reports a blind human study with 46 annotators and 289 contests across 273 queries.","source_urls":["https://cohere.com/blog/rcp-ndcg"]},{"text":"Cohere reports RCP-nDCG selected the system reviewers preferred in 77% of contests, versus 52% for conventional nDCG.","source_urls":["https://cohere.com/blog/rcp-ndcg"]}],"mechanism":"The mechanism combines five yes/no rubric questions for each document with pairwise comparisons among documents; Cohere says the comparisons establish ordering while calibration puts relevance on a scale shared across queries.","why_now":"These reported results make the evaluation method—not a newly launched search product—the material change.","implications":["The first implication is that retrieval teams may need to test whether their offline benchmarks undercount relevant results beyond their existing qrels."],"evidence_boundary":"The evidence boundary is that these are Cohere-reported findings from its study, not independent replication; rubric design, judge calibration, task selection, and sampling may affect the result.","watch_conditions":["Watch for independent evaluations across other corpora and for reproducible details of the rubric and calibration procedure."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied, so this is a distinct record for Cohere’s RCP-nDCG@10 method and its reported validation."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/rcp-ndcg-10-a-new-approach-to-enterprise-1059c0/","markdown":"https://newruntime.com/posts/rcp-ndcg-10-a-new-approach-to-enterprise-1059c0.md","json":"https://newruntime.com/posts/rcp-ndcg-10-a-new-approach-to-enterprise-1059c0.json"}
---

# RCP-nDCG@10: A New Approach to Enterprise Retrieval Quality

## Retrieval answer

Cohere’s thesis is that retrieval benchmarks can mis-score systems when their relevance labels omit useful documents.

Cohere’s thesis is that retrieval benchmarks can mis-score systems when their relevance labels omit useful documents. The company introduces Rubric-Calibrated Preferences nDCG@10 (RCP-nDCG@10), which evaluates retrieved documents against a shared rubric with a calibrated AI judge rather than treating an existing answer key as complete.

The mechanism combines five yes/no rubric questions for each document with pairwise comparisons among documents; Cohere says the comparisons establish ordering while calibration puts relevance on a scale shared across queries. In its blind study, 46 annotators assessed top-five results from NanoBEIR, BRIGHT, and ViDoRe v3. Cohere reports 289 contests across 273 queries and says RCP-nDCG selected the system reviewers preferred in 77% of contests, versus 52% for conventional nDCG. These reported results make the evaluation method—not a newly launched search product—the material change.

The first implication is that retrieval teams may need to test whether their offline benchmarks undercount relevant results beyond their existing qrels. The evidence boundary is that these are Cohere-reported findings from its study, not independent replication; rubric design, judge calibration, task selection, and sampling may affect the result. Watch for independent evaluations across other corpora and for reproducible details of the rubric and calibration procedure. Source: https://cohere.com/blog/rcp-ndcg

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)