Why does the right structural prior matter more than raw model capacity?
This explores why *how* a model is organized — its architecture, representation structure, and what gets baked into it — often beats simply making the model bigger, and what the corpus says about that trade-off.
This explores why the right structural prior — the way a model is organized, what it's built to represent, and how it routes information — can matter more than sheer parameter count. The corpus keeps circling back to one uncomfortable fact: capacity and competence are not the same thing. A model can hit perfect accuracy on a task while its internal representation is fractured and brittle — all the right features are linearly decodable, yet the underlying organization is broken in ways standard metrics never see, leaving it fragile under perturbation and distribution shift Can models be smart without organized internal structure?. Raw capacity buys you the score; structure buys you robustness.
The sharpest version of the argument is that some capabilities are *bounded by structure, not size.* In-weight factual recall is provably capped by parameter count — you cannot memorize your way out with a bigger model — but a small structural change, giving the model a tool-use circuit, decouples recall from size entirely and grants effectively unbounded facts Can models store unlimited facts without growing larger?. The same lesson shows up in how you adapt models: intervening on frozen hidden representations rather than rewriting weights achieves 10–50× better parameter efficiency, because the leverage is in *where* you act, not how much you change Can editing hidden representations beat weight updates for finetuning?. And small models trained with the right preference structure (explicit negative examples via DPO) can match much larger ones on function calling — the structural signal beats the scale Can small models match large models on function calling?.
Structure also changes the shape of what a model can even represent. Swapping deterministic latent updates for stochastic ones lets a recursive reasoner hold uncertainty and explore multiple valid solutions — something no amount of capacity gives a deterministic design, which is locked into a single prediction Can stochastic latent reasoning help models explore multiple solutions?. That same stochastic prior lets reasoning scale in *width* — sampling parallel trajectories — instead of paying the serial latency of going only deeper Can reasoning systems scale wider instead of only deeper?. Even classic scaling laws bend once you fold architectural variables (hidden size, MLP-to-attention ratio, GQA) into them: the right configuration delivered 42% more throughput *and* higher accuracy under an identical training budget Can architecture choices improve inference efficiency without sacrificing accuracy?. The budget was the same; the prior did the work.
There's a flip side worth knowing: the wrong structural prior degrades capacity you already had. Training on near-impossible RLVR samples teaches degenerate shortcuts that contaminate pre-existing skills, because group-relative normalization treats lucky successes as high-value trajectories and reinforces answer-repetition over reasoning Do overly hard RLVR samples actually harm model capabilities?. And in retrieval, late-interaction scoring on compressed vectors can't tell a structural near-miss from a real match — but a small verifier reading the *full* token-token interaction map can, because it operates on richer structure rather than a squeezed summary Can verification separate structural near-misses from topical matches?. The throughline across all of these: capacity is a ceiling, but structure decides how much of that ceiling you actually reach — and a bad prior can quietly lower it.
Sources 9 notes
Models trained with SGD can contain all the linearly decodable features needed for a task while maintaining fundamentally broken internal organization. This makes them vulnerable to perturbation and distribution shift invisible to standard evaluation metrics.
A formal proof and experiments show in-weight memorization is bounded by model size, while tool-use enables unbounded factual recall through a simple circuit. In-weight finetuning also degrades general capability by overwriting prior knowledge.
ReFT learns task-specific interventions on frozen model representations rather than updating weights, with LoReFT (low-rank linear subspace variant) dramatically outperforming LoRA across reasoning, instruction-following, and NLU benchmarks while using far fewer parameters.
Small models fine-tuned via DPO on correct and incorrect function-calling examples from a large teacher model achieve high accuracy on logical and mathematical tasks. DPO's explicit negative examples directly target the rigid output format failures where SFT alone underperforms.
GRAM replaces deterministic latent updates with stochastic sampling, enabling models to represent distributions over solutions rather than single predictions. This allows handling of ambiguous problems and multiple valid strategies that deterministic designs cannot represent.
GRAM shows that stochastic latent transitions enabling parallel trajectory sampling sidestep the serial latency cost of depth-only scaling. Width matches token-level parallelism benefits: independent paths sample the solution space without variance inflation.
Augmenting scaling laws with hidden size, MLP-to-attention ratio, and GQA configuration enables architecture optimization for inference. Optimized models achieved up to 2.1% higher accuracy and 42% greater throughput than LLaMA-3.2 under identical training budgets.
Training on nearly-impossible problems causes models to learn degenerate shortcuts rather than genuine reasoning, and these shortcuts contaminate pre-existing capabilities. Group-relative normalization treats rare accidental successes as high-advantage trajectories, reinforcing answer repetition and computation-skipping instead of sound reasoning patterns.
A two-stage pipeline—pooled-cosine recall followed by a small Transformer verifier operating on token-token similarity maps—reliably rejects structural near-misses that MaxSim-style late interaction cannot. The verifier succeeds because it operates on full token interaction patterns rather than compressed vectors.