How do language models perform syllogistic reasoning internally?
Does formal symbolic reasoning exist as a distinct neural circuit in LLMs, or is it inevitably contaminated by world knowledge associations? Understanding the mechanism could reveal whether pure logical reasoning is separable from semantic inference.
Mechanistic analysis of syllogistic inference reveals a three-stage reasoning mechanism:
- Naive recitation — the model begins by reciting information from the first premise
- Middle-term suppression — duplicated middle-term information is suppressed (e.g., in "All A are B; All B are C," the shared term B is suppressed)
- Mediation — mover attention heads transfer information to derive the valid conclusion, connecting A to C through the suppressed B
This circuit is content-independent — it operates on symbolic variables, not on the specific content of premises. When tested on schemes instantiated with commonsense knowledge, the same mechanism is still necessary. But additional attention heads encoding contextualized world knowledge contaminate the formal circuit, creating belief bias: conclusions that align with real-world knowledge are easier to derive than those that don't.
The contamination scales with model size: larger models show more complex attention head contributions, suggesting increasing interference from world knowledge. This is precisely the opposite of what you might hope — scaling doesn't purify the reasoning circuit, it adds more contamination from richer world knowledge.
The circuit is sufficient and necessary for all unconditionally valid syllogistic schemes where the model achieves ≥60% accuracy. For schemes with lower accuracy, the circuit alone is insufficient — suggesting these harder schemes require additional mechanisms the model hasn't developed.
Cross-architecture compatibility: similar suppression mechanism patterns and information flow appear across GPT-2, Pythia, Llama, and Qwen families. The reasoning mechanism is architecturally general, not model-specific.
This provides mechanistic evidence for Do large language models reason symbolically or semantically?: the model has a formal reasoning circuit, but it is inherently contaminated by semantic associations. Pure formal reasoning and world knowledge are not cleanly separable — they share neural substrate.
Inquiring lines that use this note as a source 21
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- How should emotional states integrate into symbolic reasoning systems?
- What circuit mechanisms produce belief bias in syllogistic reasoning?
- Can we distinguish between semantic and symbolic reasoning in language models?
- What makes symbolic operations different from general knowledge questions?
- How do LLMs access and draw on the same shared symbolic universe as humans?
- Can LLMs translate between natural language and formal logic faithfully?
- Why can LLMs interpret formal logic better than they generate it?
- How does semantic reasoning differ from symbolic reasoning in language models?
- How does in-context semantic reasoning differ from symbolic reasoning in concept fusion?
- Do reflection tokens and symbolic tokens serve different roles in reasoning?
- Why does premise ordering shift syllogistic reasoning performance by over 30 percent?
- Can language models perform purely symbolic reasoning when semantics are removed?
- How do knowledge and reasoning circuits interfere in the same neural network?
- Why does augmenting symbolic reasoning outperform replacing it entirely?
- Can language models perform genuine symbolic reasoning without semantic grounding?
- Why might social reasoning work differently than formal logical reasoning?
- What makes social reasoning fundamentally different from formal logical reasoning?
- How do LLMs lose information when translating natural language to formal logic?
- How does neuro-symbolic design differ from pure LLM reasoning?
- How do semantic and symbolic reasoning capabilities differ in language models?
- How should we rethink the symbolism versus connectionism debate in light of LLMs?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do large language models reason symbolically or semantically?
Can LLMs follow explicit logical rules when those rules contradict their training knowledge? Testing whether reasoning operates independently of semantic associations reveals what computational mechanisms actually drive LLM multi-step inference.
the mechanistic explanation: formal circuits exist but are contaminated by semantic attention heads
-
How much does the order of premises actually matter for reasoning?
When you rearrange the order of logical premises in a deduction task, does it change how well language models can solve it? This tests whether LLMs reason abstractly or process input sequentially.
ordering sensitivity may reflect the recitation stage: the circuit begins by reciting the first premise, so which premise is first affects the reasoning path
-
Why does reasoning training help math but hurt medical tasks?
Explores whether reasoning and knowledge rely on different network mechanisms, and why training one might undermine the other across different domains.
belief bias contamination is one mechanism for knowledge-reasoning interference
-
Which sentences actually steer a reasoning trace?
Can we identify which sentences in a reasoning trace have outsized influence on the final answer? Three independent methods converge on a surprising answer about planning and backtracking.
both findings reveal sparse mechanistic structure in reasoning: syllogistic circuits show specific attention heads performing suppression and mediation at the circuit level, while thought anchors show specific sentences dominating causal influence at the trace level; the recitation stage's attentional selectivity is the circuit-level mechanism underlying sentence-level anchor effects
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
- Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
- Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
- Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning
- LLM Reasoning Is Latent, Not the Chain of Thought
- Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
- Logical Reasoning in Large Language Models: A Survey
- How do Transformers Learn Implicit Reasoning?
Original note title
syllogistic reasoning circuits use a three-stage mechanism — recitation suppression mediation — contaminated by world knowledge bias