What makes language an effective parameterization for procedural knowledge?
This explores why natural language — rather than formal logic, code, or fixed parameters — turns out to be a good medium for storing and applying 'how-to' knowledge (the steps and strategies behind reasoning), and where that medium hits its limits.
This explores why language works so well as a container for procedural knowledge — the transferable 'how to do it' as opposed to the memorized 'what is true.' The most direct evidence comes from a study of five million pretraining documents, which found that when models reason, they lean on broad procedural patterns scattered across many diverse sources, while factual recall depends on narrowly memorizing the specific document that holds the answer Does procedural knowledge drive reasoning more than factual retrieval?. The implication is that language is effective for procedure precisely because procedures are *generalizable* — a method for solving one problem, described in words, transfers to others — whereas facts are not. Language captures the reusable shape of a method, not just the instance.
What makes language especially well-suited is that it keeps semantic richness and structure together. When researchers tried replacing natural language with fully formal symbolic logic, accuracy dropped — full formalization throws away meaning that the words carried. But pure language alone lacks rigor. The sweet spot was *partial* augmentation: enrich natural language with selective symbolic scaffolding and you get gains of several percent over either extreme Why does partial formalization outperform full symbolic logic?. Language is the substrate flexible enough to hold both the fuzzy semantics and the inserted structure at once.
The contrast that sharpens all this is code. Code is an even more powerful parameterization for *some* procedural knowledge because it's simultaneously executable, inspectable, and stateful — an agent can run a policy, check its progress, and carry state across steps Can code become the operational substrate for agent reasoning?. This points to language's real weakness: it can *describe* a procedure perfectly while being unable to *execute* it at scale. Apparent 'reasoning collapses' in large models turn out to be execution failures, not comprehension failures — the model knows the algorithm but can't reliably run it step after step in text alone, and tool access dissolves the supposed cliff Are reasoning model collapses really failures of reasoning?. So language parameterizes the *knowledge* of a procedure better than it parameterizes the *running* of one.
A second cluster of work shows how to compensate by giving language explicit structure from the outside. Embedding language inside algorithms — LLM Programs that hide step-irrelevant context and treat reasoning as modular sub-tasks — lets the procedure stay in words while the control flow is managed externally Can algorithms control LLM reasoning better than LLMs alone?. Cognitive tools push the same idea: wrapping reasoning operations in sandboxed, isolated calls lifted GPT-4.1 on competition math from 27% to 43% with no extra training, because modularity enforces the operation isolation that loose prose can't guarantee Can modular cognitive tools unlock reasoning without training?. And procedural knowledge written in language can be *accumulated*: the ACE framework treats contexts as evolving playbooks, growing them through incremental curation rather than rewrites, so hard-won procedural detail isn't compressed away over time Can context playbooks prevent knowledge loss during iteration?.
The quietly important caveat: language can only re-activate and reorganize procedural knowledge a model already has — it cannot inject what was never learned. Prompting works entirely within the training distribution, so no clever phrasing supplies a missing method Can prompt optimization teach models knowledge they lack?. Read together, the corpus suggests language is an effective parameterization for procedural knowledge because procedures are transferable and language preserves their meaning-plus-structure better than rigid formalism — but it's a parameterization for *eliciting and composing* know-how, not for executing it or creating it from nothing.
Sources 8 notes
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
QuaSAR and Logic-of-Thought both achieve 4-8% accuracy gains by enriching natural language with selective symbolic elements rather than replacing it. Full formalization loses semantic information; pure language lacks structure. Augmentation preserves both.
Research shows code uniquely enables agents to externalize reasoning, execute policies, model environments, and verify progress through its simultaneous executability, inspectability, and statefulness across task steps.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Four cognitive tools implemented as sandboxed LLM calls improved GPT-4.1 on AIME2024 from 26.7% to 43.3% without any RL training. Modularity enforces operation isolation that pure prompting cannot guarantee, eliciting pre-existing reasoning capability.
The ACE framework treats contexts as evolving playbooks using generation-reflection-curation loops rather than full rewrites. This prevents knowledge loss from compression and detail erosion, achieving +10.6% on agentic tasks and +8.6% on finance without labeled supervision.
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.