Can proper scoring rules restore model calibration without sacrificing accuracy?
This explores whether adding a proper scoring rule (like the Brier score) to training can fix a model's overconfidence without dragging down its accuracy — and what the corpus says about why calibration breaks in the first place.
This explores whether proper scoring rules can repair calibration without an accuracy tax — and the corpus has a surprisingly clean answer at the center, surrounded by reasons the problem exists at all. The cleanest result: binary correctness rewards actively *teach* models to bluff, because a confident wrong answer is penalized no more than an uncertain one — so the model learns that maximum confidence is always the safe bet. Adding the Brier score as a second reward term doesn't just patch this; it mathematically guarantees that accuracy and calibration get optimized together, with no trade-off between them Does binary reward training hurt model calibration?. So the literal answer to the question is yes — and the reason it works is that a proper scoring rule changes what 'winning' means during training, rather than bolting calibration on afterward.
The corpus also shows you don't always need an external scoring signal. One approach uses the model's *own* answer-span confidence to rank its reasoning traces, building synthetic preferences that sharpen step-by-step reasoning while reversing the calibration damage that standard RLHF inflicts — no human labels, no external verifier required Can model confidence work as a reward signal for reasoning?. Read together, these two say something useful: calibration and capability aren't opposed resources you must trade between. The trade-off people assume is real is largely an artifact of *which objective* you optimized.
That 'which objective' framing turns out to be the deeper story. Calibration isn't one dial you turn — it's a failure signature that points in opposite directions depending on training. Reasoning-trained models *under*-abstain and over-answer because abstention earns no reward; safety-trained models *over*-abstain and refuse harmless questions. There's no single axis to fix, because the direction of miscalibration is inherited from whatever objective dominated Does training objective determine which direction models fail at abstention?. A proper scoring rule helps precisely because it reintroduces the missing penalty — it makes the objective care about the thing the original reward ignored.
Why this matters beyond the math: confident wrong answers are nearly invisible to the metrics teams actually watch. Aggregate accuracy stays high while fluent, self-assured errors concentrate in the rare, high-harm cases — medical triage, legal reading, financial planning — where surface heuristics collide with unstated constraints Why do confident wrong answers hide in standard accuracy metrics?. Calibration is the thing standard evaluation can't see, which is exactly why a reward that explicitly prices confidence is worth the trouble. And confidence isn't just a safety nicety — well-calibrated confidence predicts robustness, with high-confidence models resisting prompt rephrasing while low-confidence ones swing wildly on cosmetic changes Does model confidence predict robustness to prompt changes?.
One caution the corpus quietly raises: don't confuse a number that *looks* settled with one that's *reliable*. Setting temperature to zero produces the same output every time, but that's one fixed draw from the distribution, not evidence the model is well-calibrated Does setting temperature to zero actually make LLM outputs reliable?. Proper scoring rules work on the probabilities themselves — they're a real fix, not a cosmetic one, which is what separates them from the determinism trap.
Sources 6 notes
Binary correctness rewards incentivize high-confidence guessing because they don't penalize confident wrong answers. Adding the Brier score as a second reward term mathematically guarantees joint optimization of accuracy and calibration without trade-off.
RLSF uses answer-span confidence to rank reasoning traces, creating synthetic preferences that strengthen step-by-step reasoning while reversing RLHF's calibration degradation—without requiring human labels or external verifiers.
Reasoning-trained models under-abstain and overanswer because abstention is unrewarded, while safety-trained models over-abstain and refuse benign questions. This reveals calibration is not a single fixable axis but a characteristic failure signature that depends on which objective dominated training.
Medical triage, legal interpretation, and financial planning show a consistent pattern: surface heuristics conflict with unstated constraints, producing fluent confident errors that concentrate in rare cases where harm occurs. Aggregate accuracy masks these failures because overall performance looks strong.
ProSA found that when models are highly confident, they resist prompt rephrasing; low confidence causes major output swings. Larger models, few-shot examples, and objective tasks all correlate with higher confidence and greater robustness.
Fixed seeds and zero temperature replicate the same output repeatedly, but that output remains one draw from the model's probability distribution. McDonald's omega testing across 100 repetitions reveals that consistency does not equal reliability.