Does model scaling improve knowledge storage faster than reasoning ability?
This explores whether bigger models gain factual knowledge more readily than they gain reasoning skill — and the corpus reframes the premise: scaling mostly surfaces reasoning that's already latent, while knowledge and reasoning turn out to depend on different mechanisms entirely.
This reads the question as asking whether scaling buys you knowledge storage and reasoning ability at different rates. The corpus doesn't measure that race head-on, but it converges on something more interesting: knowledge and reasoning aren't even the same kind of thing inside a model, so they don't scale on the same curve. The sharpest evidence is the split between how the two are stored. Factual recall depends on narrow, document-specific memorization — the model essentially needs to have seen the target fact — while reasoning draws on broad, transferable procedural knowledge spread across many unrelated documents Does procedural knowledge drive reasoning more than factual retrieval?. That means adding parameters and data reliably packs in more facts (memorization scales cleanly), but reasoning improvement is a different game entirely.
And the surprising part of that game is that scaling may not be 'adding' reasoning so much as 'unlocking' it. Multiple independent methods — RL steering, critique fine-tuning, decoding tweaks, feature steering — all elicit reasoning that already sits in base model activations Do base models already contain hidden reasoning ability?. The bottleneck is elicitation, not capacity. So the intuition behind the question (knowledge fills up faster than reasoning grows) partly dissolves: reasoning capability isn't slowly accumulated with scale, it's present and waiting for the right training signal to select it. What separates a strong reasoner from a weak one is often the training regime, not raw size or inference budget — reasoning models beat non-reasoning ones no matter how much extra compute you throw at the latter Can non-reasoning models catch up with more compute?.
There's also a real question of whether what looks like 'more reasoning ability' is reasoning at all. Chain-of-thought degrades predictably outside the training distribution, producing fluent but logically broken steps — the model imitates the form of reasoning without the underlying logic Does chain-of-thought reasoning actually generalize beyond training data?. Failures cluster at unfamiliar instances rather than at some complexity ceiling, suggesting models fit instance-level patterns instead of learning general algorithms Do language models fail at reasoning due to complexity or novelty?. Seen this way, some of the 'reasoning' that scaling appears to deliver is really broader memorized coverage — which looks a lot like knowledge storage wearing a reasoning costume.
A further wrinkle: even when a model genuinely knows how to reason, it can fail to execute. Collapses on long multi-step problems are often execution-bandwidth limits, not reasoning limits — give the model tools and it solves problems past the supposed cliff Are reasoning model collapses really failures of reasoning?. And reasoning accuracy drops sharply just from longer inputs, well below the context window, independent of language-modeling quality Does reasoning ability actually degrade with longer inputs?. These ceilings don't move much with scale; they move with architecture and deployment.
So the honest answer the corpus points to: model scaling does appear to bank factual knowledge steadily because memorization is what scale is good at, but reasoning gains are less about parameters and more about elicitation, training protocol, execution machinery, and staying in-distribution. The two genuinely advance at different rates — but not for the reason the question assumes. The reasoning capacity is largely already there; scaling and post-training mostly change how reliably you can reach it.
Sources 7 notes
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Reasoning models persistently outperform non-reasoning models regardless of inference budget because training instills a reasoning protocol that makes additional tokens productive. The gap is fundamentally about deployment mechanisms and training structure, not raw capability.
DataAlchemy experiments show CoT fails systematically under distributional shifts in task, length, and format. Models produce fluent but logically inconsistent reasoning — imitating reasoning form without valid underlying logic.
LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.