How does the functional separation of knowledge and reasoning affect adaptation methods?
This explores what happens when you treat 'what a model knows' and 'how a model reasons' as two separate things — and how that split changes the way we adapt, fine-tune, or train models.
This explores what happens when you treat a model's stored knowledge and its reasoning process as two separable capabilities — and how that separation reshapes adaptation methods. The corpus makes a striking case: a lot of what we call 'teaching a model to reason' is really just teaching it *when and how to organize what it already has*, not pouring in new ability. A 1.5B model with only LoRA format-adaptation matched far larger RL-trained models, suggesting RL teaches output organization rather than new facts Can small models reason well by just learning output format?. That lands alongside the broader finding that base models already carry latent reasoning, which minimal training merely unlocks Do base models already contain hidden reasoning ability?, and the sharper claim that RL post-training selects *when* to reason rather than installing *how* Does RL post-training create reasoning or just deploy it?. If reasoning and knowledge live in different places, adaptation can target one without disturbing the other.
The biggest practical payoff is escaping catastrophic forgetting. Fast-Slow Training routes durable task lessons into optimized prompts (fast, textual) while barely touching weights (slow, parametric), reaching the same performance up to 3x faster with far less forgetting — reframing forgetting as a *misallocation* problem rather than an inevitable cost Can splitting adaptation into two channels reduce forgetting?. The two-channel idea recurs: separate a decomposer (planning) from a solver (execution), and the planning skill generalizes across domains while solving does not — interference between the two was quietly hurting both Does separating planning from execution improve reasoning accuracy?. Cognitive tools push this further, isolating individual reasoning operations as modular calls and lifting GPT-4.1 on hard math with zero RL Can modular cognitive tools unlock reasoning without training?, while LLM Programs wrap models in explicit algorithms that hand each call only the context it needs Can algorithms control LLM reasoning better than LLMs alone?.
There's a cautionary thread too. Fine-tuning that doesn't respect the separation can quietly sever reasoning from answers: chains become *performative* — they look like reasoning but stop actually driving the output, surviving paraphrasing and even truncation unchanged Does fine-tuning disconnect reasoning steps from final answers?. So the question isn't only whether to separate the two capabilities, but whether your adaptation method preserves the *causal* link between them.
The separation also opens lighter-touch alternatives to weight surgery entirely. Decoupled RL lets a model learn to route between deep thinking and quick answers without difficulty labels Can models learn when to think versus respond quickly?; decoding-level penalties fix structural failures like wandering and premature path-switching with no fine-tuning at all Why do reasoning models abandon promising solution paths?; and memoryless, Markov-style reasoning contracts a problem so each step depends only on the current state, shedding accumulated history Can reasoning systems forget history without losing coherence?. The counterpoint comes from RLAG, which argues that when you genuinely *do* need to internalize new domain knowledge, rewarding explanation quality embeds it more coherently than supervised fine-tuning Can reinforcement learning embed domain knowledge more effectively than supervised fine-tuning?.
The thing worth walking away with: once you stop treating knowledge and reasoning as one undifferentiated blob, most of adaptation's worst pathologies — forgetting, faithfulness collapse, planning-execution interference — start looking like symptoms of mixing two things that wanted to be adapted separately.
Sources 12 notes
A 1.5B parameter model with LoRA-only post-training matched larger full-parameter RL models on reasoning tasks, suggesting RL teaches output format organization rather than new factual knowledge. This efficiency indicates reasoning and knowledge storage are separable capabilities.
Five independent mechanisms—RL steering, critique fine-tuning, decoding changes, SAE feature steering, and RLVR—all elicit reasoning already present in base model activations. Post-training selects rather than creates reasoning; the bottleneck is elicitation, not capability acquisition.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
Fast-Slow Training routes task-specific lessons into optimized prompts while keeping parameter updates minimal, reaching equivalent performance 1.4–3x faster with substantially less catastrophic forgetting and plasticity loss, demonstrating that forgetting is a misallocation problem rather than an inherent cost.
Modular architectures with separate decomposer and solver models outperform monolithic LLMs, with decomposition ability transferring across domains while solving ability does not. The separation prevents planning-execution interference and produces more generalizable skills.
Four cognitive tools implemented as sandboxed LLM calls improved GPT-4.1 on AIME2024 from 26.7% to 43.3% without any RL training. Modularity enforces operation isolation that pure prompting cannot guarantee, eliciting pre-existing reasoning capability.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Three faithfulness tests show fine-tuned models generate reasoning chains that less reliably influence final outputs. Early termination, paraphrasing, and filler substitution all produce invariant answers more often after fine-tuning, suggesting reasoning becomes performative rather than functional.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
Reasoning LLMs exhibit two reinforcing failures: wandering (invalid exploration) and underthinking (premature path-switching). Decoding-level interventions like thought-switching penalties improve accuracy without fine-tuning, suggesting viable solutions exist but are abandoned prematurely.
Atom of Thoughts decomposes problems into DAGs and contracts them iteratively, ensuring each state depends only on the current problem—not prior steps. This memoryless approach eliminates historical baggage that bloats reasoning while maintaining answer equivalence.
RLAG rewards both answer accuracy and explanation rationality by cycling between augmented and unaugmented generation, progressively internalizing coherent knowledge structures. This outperforms SFT because it prioritizes reasoning quality over token-level correctness.