These estimates address different questions: expected ability on the task and correctness of a specific answer.
Cohere reports training 7B and 30B models on 30,000 math problems and evaluating them on 1,000 held-out problems plus three question-answering benchmarks. On BigMathDigits, the 7B model’s expected calibration error fell from 0.059 for the strongest single-confidence baseline to 0.028. The source also reports calibration-error reductions on GPQA, TriviaQA, and SimpleQA. These are benchmark results from Cohere’s experiments, not evidence that confidence-guided verification or abstention works in deployment.