{"type":"raw_signal","id":"rethinking-confidence-in-reasoning-models-20e9ea","slug":"rethinking-confidence-in-reasoning-models-20e9ea","title":"Rethinking Confidence in Reasoning Models","description":"Cohere describes CALIBER, a training method that asks a reasoning model to report two confidence estimates: its expected chance of solving a problem before reasoning and its estimated chance that the particular answer is correct afterward.","retrieval_nugget":"Cohere describes CALIBER, a training method that asks a reasoning model to report two confidence estimates: its expected chance of solving a problem before reasoning and its estimated chance that the particular answer is correct afterward.","observed_at":"2026-10-20","record_date":"2026-10-20","date_kind":"observed_at","why_it_matters":"Cohere reports training 7B and 30B models on 30,000 math problems and evaluating them on 1,000 held-out problems plus three question-answering benchmarks. On BigMathDigits, the 7B model’s expected calibration error fell from 0.059 for the strongest single-confidence baseline to 0.028. The source also reports calibration-error reductions on GPQA, TriviaQA, and SimpleQA. These are benchmark results ","novelty":"notable","verification_level":"source-linked","signal_type":"research","source_platform":"cohere.com","topics":[],"entities":["Cohere","CALIBER","BigMathDigits","GPQA","TriviaQA","SimpleQA"],"related_patterns":[],"novelty_label":"new_fact","what_changed":"Cohere’s confidence-in-reasoning-models research is a distinct topic from prior coverage of controlling reasoning effort in LLMs; the archive contains no details of this confidence study.","prior_refs":["memory:54946dd1-6f16-8f81-aed9-177bd3373c77"],"source_url":"https://cohere.com/blog/rethinking-confidence-in-reasoning-models","source_urls":["https://cohere.com/blog/rethinking-confidence-in-reasoning-models"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"signal:rethinking-confidence-in-reasoning-models-20e9ea","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/signals/rethinking-confidence-in-reasoning-models-20e9ea/","markdown":"https://newruntime.com/signals/rethinking-confidence-in-reasoning-models-20e9ea.md","json":"https://newruntime.com/signals/rethinking-confidence-in-reasoning-models-20e9ea.json"},"next_reads":[{"type":"collection","path":"/signals/","url":"https://newruntime.com/signals/","title":"Raw Signals","reason":"Open every source-linked raw signal.","media_type":"text/html","json_url":"https://newruntime.com/signals.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}