---
type: "post"
id: "new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b"
slug: "new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b"
title: "New SciFullBench benchmark provides full-paper queries and candidates from ICLR, NeurIPS, ACL, EMNLP."
description: "SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates."
retrieval_nugget: "SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates."
published_at: "2026-10-20"
updated_at: "2026-10-20"
record_date: "2026-10-20"
date_kind: "published_at"
topics: ["retrieval"]
source_urls: ["https://arxiv.org/abs/2507.10057"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:5b77c8c8814596b5c45f31156f5cd1ed6925768c05391a028c230a53316272a0","reviewed_at":"2026-10-20T08:05:15.041Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Full-paper retrieval becomes more testable when benchmarks preserve document context and the retrieval process can exploit relations among candidate papers.","observed_facts":[{"text":"The abstract says SciFullBench provides both complete and segmented contexts of full papers for queries and candidates.","source_urls":["https://arxiv.org/abs/2507.10057"]},{"text":"The abstract describes CoR as decomposing query papers into aspect-specific views and matching them against segmented candidate papers.","source_urls":["https://arxiv.org/abs/2507.10057"]},{"text":"The abstract reports that CoR significantly outperforms existing retrieval baselines.","source_urls":["https://arxiv.org/abs/2507.10057"]}],"mechanism":"CoR uses aspect-specific matching to identify candidates, promotes top-ranked results as new queries to iteratively expand search, then aggregates the retrieval tree in post-order.","why_now":"The paper identifies abstract-only representations as sparse summaries that miss dynamic relations among relevant papers.","implications":["Researchers can evaluate retrieval systems against complete and segmented paper contexts.","Agent builders can consider iterative candidate expansion and aggregation beyond one-shot similarity."],"evidence_boundary":"This account rests on the arXiv abstract; it does not establish the magnitude, robustness, or practical generality of the reported gains.","watch_conditions":["Check the paper's dataset composition, split design, metrics, and numerical comparisons before treating the reported advantage as a deployment signal."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied; this benchmark and retrieval method therefore start a distinct analytical record."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b/","markdown":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b.md","json":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b.json"}
related_patterns: ["scientific-agents-need-artifact-evidence"]
---

# New SciFullBench benchmark provides full-paper queries and candidates from ICLR, NeurIPS, ACL, EMNLP.

## Retrieval answer

SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates.

SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates. The paper says the benchmark provides both complete and segmented contexts. Its proposed Chain of Retrieval (CoR) decomposes a query paper into aspect-specific views, matches those views against candidate-paper segments, and iteratively promotes top-ranked results as new queries. The resulting tree is aggregated from descendants toward parent nodes.

Thesis: Full-paper retrieval becomes more testable when benchmarks preserve document context and the retrieval process can exploit relations among candidate papers. Mechanism: CoR uses aspect-specific matching to identify candidates, promotes top-ranked results as new queries to iteratively expand search, then aggregates the retrieval tree in post-order. The process replaces a single similarity comparison with staged matching, candidate expansion, and aggregation. Why now: The paper identifies abstract-only representations as sparse summaries that miss dynamic relations among relevant papers. These mechanisms make SciFullBench relevant to the quality of research agents that must discover evidence across full documents, rather than merely match short summaries.

Implication: Researchers can evaluate retrieval systems against complete and segmented paper contexts. Agent builders can consider iterative candidate expansion and aggregation beyond one-shot similarity. The authors report that CoR significantly outperforms existing retrieval baselines, but the supplied abstract gives no benchmark size, baseline names, scores, or evaluation protocol. Evidence boundary: This account rests on the arXiv abstract; it does not establish the magnitude, robustness, or practical generality of the reported gains. Watch condition: Check the paper's dataset composition, split design, metrics, and numerical comparisons before treating the reported advantage as a deployment signal. Source: https://arxiv.org/abs/2507.10057

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Retrieval](https://newruntime.com/topics/retrieval.md): Explore the retrieval topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)