Do self-revision tokens measurably degrade reasoning accuracy in scaled models?
This explores whether the tokens a model spends second-guessing and rewriting its own reasoning actually make it less accurate — and the corpus says the act of self-revision is more often a liability than a fix.
This explores whether self-revision tokens — the part of a chain where a model reconsiders and rewrites its own work — degrade reasoning accuracy, especially as chains get longer. The corpus is unusually direct here: in o1-style reasoning models, most revisions retain the wrong answer rather than correcting it, and longer chains stacked with more revisions correlate with *lower* accuracy, not higher Does self-revision actually improve reasoning in language models?. Smaller models are the worst offenders — they routinely flip a correct answer to an incorrect one mid-revision.
The sharper insight is *why* this happens, and it isn't the revising itself — it's who is doing the revising. When an external critic guides the revision, accuracy improves; when a model critiques its own uncertain output, it tends to amplify confidence in the wrong answer instead of catching it Does revising your own reasoning actually help or hurt?. That reframes your question: self-revision tokens degrade reasoning because a model's internal self-assessment is a poor error detector. This lines up with a formal ceiling — reliable self-correction needs something external to validate the fix, and metacognition alone can't close that generation-verification gap What stops large language models from improving themselves?.
There's a broader pattern the corpus points to: more reasoning tokens are not free. Accuracy rises, peaks, then falls as thinking tokens scale — one benchmark dropped from 87% to 70% as tokens climbed from ~1,100 to ~16K, with models overthinking easy problems Does more thinking time always improve reasoning accuracy?. Self-revision is one of the most expensive token categories, so it sits right in the zone where extra length actively hurts.
What makes this counterintuitive is that not all tokens carry equal weight. Reasoning chains internally rank tokens by function, preferentially preserving symbolic computation while grammar and meta-discourse get pruned first Which tokens in reasoning chains actually matter most? — and only about 20% of tokens, the high-entropy 'forking' decisions, actually drive learning Do high-entropy tokens drive reasoning model improvements?. Self-revision tends to be meta-discourse layered on top, not the load-bearing computation. Stranger still, models trained on deliberately corrupted reasoning traces perform comparably to those trained on correct ones, suggesting traces often work as computational scaffolding rather than genuine self-checking Do reasoning traces need to be semantically correct?.
The doorway worth walking through: if verbalized self-revision is mostly noise, what's the alternative? One line of work scales test-time compute in latent space — iterating hidden states without generating visible thinking tokens at all Can models reason without generating visible thinking tokens?, hinting that the real reasoning gains may come from quieter computation rather than from a model talking itself into and out of answers.
Sources 8 notes
Evidence from QwQ, R1, and LIMO shows most revisions retain wrong answers rather than correcting them. Smaller models frequently switch correct answers to incorrect during revision, and longer chains with more revisions correlate with lower accuracy.
Revision guided by external models improves accuracy, but a model revising its own uncertain output typically amplifies confidence in wrong answers rather than correcting them. The revision source, not the revision act itself, determines the outcome.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
Increasing thinking tokens from ~1,100 to ~16K reduced benchmark accuracy from 87.3% to 70.3%, revealing a non-monotonic relationship where models overthink easy problems and underthink hard ones.
Greedy likelihood-preserving pruning reveals six functional token categories; symbolic computation tokens are preferentially preserved while grammar and meta-discourse are pruned first. Student models trained on these pruned chains outperform those trained on frontier-model compression.
Only ~20% of tokens exhibit high entropy as pivotal reasoning decision points; RLVR primarily adjusts these forking tokens. Training exclusively on them matches or exceeds full-gradient performance, revealing that the minority carries the learning signal.
Models trained on systematically irrelevant traces maintain solution accuracy and sometimes improve out-of-distribution generalization, suggesting traces function as computational scaffolding rather than meaningful reasoning steps.
Multiple architectures—depth-recurrent models, Heima, and Coconut—demonstrate that test-time compute scales through hidden state iteration rather than token generation. This suggests verbalization is a training artifact, not a reasoning requirement.