SYNTHESIS NOTE
Topics›Training Fine Tuning›this note

Does procedural knowledge drive reasoning more than factual retrieval?

Explores whether models learn reasoning through general procedures across diverse documents rather than memorizing specific facts. This matters for understanding what pretraining data actually teaches models to reason.

Synthesis note · 2026-02-22 · sourced from Training Fine Tuning

The "Procedural Knowledge in Pretraining Drives Reasoning" paper analyzes which pretraining documents most influence LLM reasoning by ranking 5 million documents by their influence on model completions. The finding: the approach to reasoning that models use is unlike retrieval. For reasoning tasks, positively influential documents contain procedural knowledge — descriptions of how to get to a solution — rather than the specific facts needed for the answer.

Three contrasts with factual recall:

  1. Generality: models rely on a broader, more general set of documents when reasoning than when answering factual questions. Factual recall draws on a narrow set of documents containing the target fact. Reasoning draws on a diffuse set of documents performing similar procedures.

  2. Transferability: documents have similar influence on reasoning queries that require applying the same procedure to different numbers. The procedural knowledge transfers across specific instances — it's the method, not the content, that the model has learned.

  3. Reliance distribution: the model needs to see factual information more often (across more documents) to memorize it, while procedural patterns can be learned from fewer but more diverse demonstrations.

This connects to the knowledge/reasoning layer separation. Since Why does reasoning training help math but hurt medical tasks?, the procedural knowledge finding provides the data-level explanation for the architectural finding: lower layers store memorized facts (requiring document-specific exposure), while higher layers encode procedural strategies (learnable from general demonstrations).

The implication for training data curation: reasoning capability benefits more from diverse demonstrations of procedures than from exhaustive factual coverage. Quality and diversity of reasoning demonstrations may matter more than volume for building reasoning capability — consistent with Can models improve themselves on tasks without verifiable answers?.

Inquiring lines that read this note 170

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What happens to knowledge when intelligence becomes tokenized like a commodity? Can prompt-based context override biases that were embedded during pretraining? Why does adding new knowledge through fine-tuning degrade existing capabilities? Why don't LLMs reliably translate capability into accurate outputs? Why do stronger reasoning capabilities create tradeoffs with instruction following? Is reasoning capability latent in base models or created by post-training? Can models improve accuracy without degrading reasoning quality? What causes reasoning models to fail or wander off track? How should systems decide whether to retrieve or reason alone? Why can't prompting alone inject genuinely new knowledge into models? How much do training data properties shape model reasoning? Do language models reason through causal mechanisms or semantic associations? How does reasoning length affect model performance across different tasks? Do reasoning traces faithfully reflect actual model reasoning? How do prompt design choices influence model reasoning and performance? How much does training format versus domain influence reasoning? When do semantic similarity approaches miss structural retrieval failures? Does AI assistance promote real skill development or substitute for independent learning? Can memory architectures handle ultra-long context better than attention? Can reasoning scale in latent space without tokens? Does encoded knowledge in language models actually influence their outputs? Do language models lack essential therapeutic presence and engagement? How effectively can language models perform reasoning, especially combined with symbolic methods? Why do some clarifying approaches produce understanding while others just satisfy? What capability trade-offs arise from domain specialization through fine-tuning? Can brute-force automated research substitute for iterative depth and human research intuition? Do language models develop actual world models or merely task heuristics? What training data selection strategies maximize generalization across difficulty levels? Why do embedding systems fail to capture task-relevant relationships? What enables genuine semantic understanding in language models? What training dynamics and scale trigger emergence of reasoning capabilities? What reasoning architectures enable models to solve complex problems efficiently? Does RL create genuinely new reasoning capabilities or refine existing ones? How do neural networks achieve compositional generalization at scale? How does self-revision in reasoning models affect accuracy and confidence? Why do token-level mechanisms matter for learning to reason? Do structural constraints outperform deep architectures in recommendation systems? How do prompting refinements mask underlying biases and model frequency patterns? How do agent-learned skills transfer and improve across different tasks? Can self-generated feedback reliably guide model training without ground truth? Why does memory consolidation cause performance regression in continual learning? How do spurious versus genuine rewards shape model reasoning and behavior? Is language model reasoning authentic and what causes models to reason? How should designers communicate what AI systems truly are and can do? Does chain-of-thought reasoning reveal genuine computation or imitate patterns? Can mechanistic interpretability reliably guide practical model design choices? How does the generation-verification gap limit what we can measure about AI reasoning? When should work require human-AI partnership versus full automation? How should agents manage memory granularity to improve long-term performance?

Related concepts in this collection 5

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
16 direct connections · 171 in 2-hop network ·dense cluster Open in graph ↗
Does procedural knowledge drive reasoning more t… Why does reasoning training help math but hurt med… Can models improve themselves on tasks without ver… Do base models already contain hidden reasoning ab… Do foundation models learn world models or task-sp… Can text-trained models compress images better tha…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why does reasoning transfer across different numbers but factual recall does not?How many document exposures does procedural knowledge versus factual information require?Can smaller amounts of diverse reasoning demonstrations replace exhaustive factual training data?Why do higher network layers capture procedural knowledge but lower layers store facts?
Original note title

procedural knowledge in pretraining documents drives reasoning generalization unlike factual retrieval which requires document-specific memorization