{"type":"post","id":"new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b","slug":"new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b","title":"New SciFullBench benchmark provides full-paper queries and candidates from ICLR, NeurIPS, ACL, EMNLP.","description":"SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates.","retrieval_nugget":"SciFullBench makes the evaluation object in scientific retrieval more demanding: its query and candidate contexts can be complete papers, not only abstract surrogates.","published_at":"2026-10-20","updated_at":"2026-10-20","record_date":"2026-10-20","date_kind":"published_at","topics":["retrieval"],"source_urls":["https://arxiv.org/abs/2507.10057"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:5b77c8c8814596b5c45f31156f5cd1ed6925768c05391a028c230a53316272a0","reviewed_at":"2026-10-20T08:05:15.041Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Full-paper retrieval becomes more testable when benchmarks preserve document context and the retrieval process can exploit relations among candidate papers.","observed_facts":[{"text":"The abstract says SciFullBench provides both complete and segmented contexts of full papers for queries and candidates.","source_urls":["https://arxiv.org/abs/2507.10057"]},{"text":"The abstract describes CoR as decomposing query papers into aspect-specific views and matching them against segmented candidate papers.","source_urls":["https://arxiv.org/abs/2507.10057"]},{"text":"The abstract reports that CoR significantly outperforms existing retrieval baselines.","source_urls":["https://arxiv.org/abs/2507.10057"]}],"mechanism":"CoR uses aspect-specific matching to identify candidates, promotes top-ranked results as new queries to iteratively expand search, then aggregates the retrieval tree in post-order.","why_now":"The paper identifies abstract-only representations as sparse summaries that miss dynamic relations among relevant papers.","implications":["Researchers can evaluate retrieval systems against complete and segmented paper contexts.","Agent builders can consider iterative candidate expansion and aggregation beyond one-shot similarity."],"evidence_boundary":"This account rests on the arXiv abstract; it does not establish the magnitude, robustness, or practical generality of the reported gains.","watch_conditions":["Check the paper's dataset composition, split design, metrics, and numerical comparisons before treating the reported advantage as a deployment signal."],"related_records":[],"new_branch_reason":"No prior New Runtime record is supplied; this benchmark and retrieval method therefore start a distinct analytical record."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b/","markdown":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b.md","json":"https://newruntime.com/posts/new-scifullbench-benchmark-provides-full-paper-queries-and-9c116b.json"},"related_patterns":["scientific-agents-need-artifact-evidence"],"next_reads":[{"type":"topic","path":"/topics/retrieval/","url":"https://newruntime.com/topics/retrieval/","title":"Retrieval","reason":"Explore the retrieval topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/retrieval.md","json_url":"https://newruntime.com/topics/retrieval.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}