Why does partial formalization outperform full symbolic logic?
Explores whether injecting some symbolic structure into natural language reasoning works better than completely formalizing problems. Matters because it could reveal the optimal balance between structure and semantics for LLM reasoning.
Two independent approaches converge on the same principle: injecting partial symbolic structure into natural language context outperforms both pure NL reasoning and full symbolic formalization. The key is augmentation, not replacement.
QuaSAR (Quasi-Symbolic Abstract Reasoning) guides LLMs through four steps: (1) Abstraction — identify relevant predicates, variables, constants; (2) Formalization — reformulate using a mix of symbols and NL; (3) Explanation — solve using quasi-symbolic representations; (4) Answering — extract the final answer. The model formalizes only what's relevant, keeping everything else in NL. Result: up to 8% accuracy improvement on MMLU-Redux and GSM-Symbolic, with enhanced robustness on adversarial variations.
Logic-of-Thought (LoT) takes a different path to the same destination: extract propositional logic from the input, expand via logical reasoning laws (double negation, contraposition, transitivity), translate the expanded logic back to NL, then inject as additional context alongside the original prompt. Result: +4.35% on ReClor (with CoT), +3.52% on RuleTaker (with CoT+SC), +8% on ProofWriter (with ToT).
Both approaches solve the same problem differently. Full neuro-symbolic methods (Logic-LM, LINC, SatLM) translate the ENTIRE problem to formal logic, which inevitably loses information — the LoT paper documents specific cases where "Harry is a person" and "Walden is a book" are lost during extraction, causing symbolic solvers to fail. QuaSAR and LoT avoid this by keeping the original NL context intact and adding formal elements as enrichment.
The theoretical grounding is illuminating. QuaSAR draws on Kitcher's unificationist account of explanation: explanations work by subsuming observations under recurring argument patterns through abstraction. Replacing concrete entities with abstract symbols creates reusable reasoning patterns — the same pattern can explain why objects fall AND why celestial bodies attract. This is partial formalization as cognitive tool, not as logical translation.
Since Can large language models translate natural language to logic faithfully?, full formalization is a dead end. Since Do large language models reason symbolically or semantically?, removing semantics breaks reasoning. The partial approach threads the needle: add enough structure to bypass content bias while preserving enough semantics for the model to reason.
Inquiring lines that use this note as a source 62
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- What structural constraints matter more than model depth for CF?
- How does the frame problem differ between symbolic and statistical reasoning systems?
- How should emotional states integrate into symbolic reasoning systems?
- What are the five structure types and which tasks does each one suit best?
- Can symbolic solvers rescue language models from logical reasoning failures?
- Would hybrid systems combining LLMs with symbolic solvers overcome the retraction limitation?
- Can we distinguish between semantic and symbolic reasoning in language models?
- What graph structures would enable transformational creative reasoning in LLMs?
- Do reasoning languages like Prolog follow the same two-constraint transfer pattern?
- What makes symbolic operations different from general knowledge questions?
- Can LLMs improve at metaphor if they handle decoupled semantics better?
- Why does GraphRAG prioritize corpus completeness while LogicRAG prioritizes query adaptivity?
- Why does comparison reasoning generalize better than composition reasoning?
- Why do LLMs generate logical forms without preserving semantic content?
- Which knowledge structure types best fit different query types?
- Can LLMs translate between natural language and formal logic faithfully?
- Why do LLMs struggle with negation and exception handling?
- Why can't LLMs reason from first principles or initial commitments?
- How does the distance between natural language and formal notation affect translation accuracy?
- Why can LLMs interpret formal logic better than they generate it?
- How does structural complexity affect LLM performance differently than inferential complexity?
- How does semantic reasoning differ from symbolic reasoning in language models?
- Why do language models struggle with formal logical reasoning and joins?
- Do LLMs lack architectural scaffolding for compositional reasoning?
- Why do only context-sensitive formal languages transfer effectively to natural language?
- How does structural complexity in sentences degrade LLM reasoning systematically?
- What makes structural logic correlate so strongly with contextual consistency?
- How does in-context semantic reasoning differ from symbolic reasoning in concept fusion?
- What makes tarot and periodic tables resist meaningful scientific integration?
- Do reflection tokens and symbolic tokens serve different roles in reasoning?
- Does architectural separation of induction from deduction improve exception detection?
- Why do format and structure matter more than actual content in reasoning?
- Which constraint types do reasoning models handle best?
- Can language models perform purely symbolic reasoning when semantics are removed?
- Why does augmenting symbolic reasoning outperform replacing it entirely?
- Why do LLMs struggle to translate natural language into logical formalizations?
- How does symbolic solver feedback differ from language-based self-critique?
- Does structured decomposition improve LLM reasoning in other compound tasks?
- Can LLMs successfully translate natural language into formal solver specifications?
- What makes language an effective parameterization for procedural knowledge?
- Why does augmenting natural language with formal representations outperform full formalization?
- How do deterministic symbolic solvers improve the reliability of language model reasoning?
- Does wrapping existing protocols create lowest-common-denominator abstractions that lose sharpness?
- How do progressive abstraction chains differ from branching reasoning topologies?
- What other structural limits exist at the language-formal boundary?
- Why do semi-formal templates improve verification accuracy over unstructured reasoning?
- How do LLMs translate informal prose into logically correct formal specifications?
- Can partial formal verification work without full formalization of language semantics?
- Why does moving verifier synthesis to the LLM extend verification beyond math and code domains?
- How does MaxSim reranking differ from structural verification at the token level?
- What makes structured stochasticity more effective than unstructured randomness in reasoning?
- How do LLMs lose information when translating natural language to formal logic?
- Can symbolic solvers reliably replace LLM reasoning for logical tasks?
- How do completeness scaffolds force explicit step-by-step derivation?
- Can completeness scaffolding substitute for actual code execution in reasoning?
- What makes natural language reasoning more practical than formal languages for multi-framework codebases?
- Can completeness scaffolding work for domains beyond code verification?
- How does neuro-symbolic design differ from pure LLM reasoning?
- How do semantic and symbolic reasoning capabilities differ in language models?
- What structural framework prevents LLM explanations from becoming just plausible fiction?
- What types of math proofs benefit most from proof-by-contradiction framing?
- What makes procedural knowledge in documents generalize better than facts?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can large language models translate natural language to logic faithfully?
This explores whether LLMs can convert natural language statements into formal logical representations without losing meaning. It matters because faithful translation is essential for any AI system that reasons formally or verifies specifications.
full formalization fails; partial avoids the failure
-
Do large language models reason symbolically or semantically?
Can LLMs follow explicit logical rules when those rules contradict their training knowledge? Testing whether reasoning operates independently of semantic associations reveals what computational mechanisms actually drive LLM multi-step inference.
preserving semantics is necessary; adding partial structure is sufficient
-
Can symbolic solvers fix how LLMs reason about logic?
LLMs excel at understanding natural language but fail at precise logical inference. Can pairing them with deterministic symbolic solvers—using solver feedback to refine attempts—overcome this fundamental weakness?
full symbolic offloading is the other extreme; this is the productive middle ground
-
Can structured argument prompts make LLM reasoning more rigorous?
Does requiring language models to explicitly check warrants, backing, and rebuttals—rather than reasoning freely—improve reasoning quality and catch failures that standard step-by-step prompting misses?
CQoT is another form of partial structure injection; both add formal elements without full formalization
-
Do formal language prototypes improve reasoning across different domains?
Can training language models on abstract reasoning patterns in Prolog and PDDL help them generalize to new reasoning tasks? This tests whether shared logical structures underlie seemingly different problem domains.
ProtoReasoning takes the augmentation approach at the training level: Prolog/PDDL prototypes alongside natural language, not replacing it; the 4-6% cross-domain improvement confirms that partial formal augmentation transfers better than full formalization, extending the augmentation principle from inference-time (this note) to training-time
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
- Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
- Probing Structured Semantics Understanding and Generation of Language Models via Question Answering
- Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
- Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations
- Large Language Models as Planning Domain Generators
- Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models
- From Language to Logic: A Bi-Level Framework for Structured Reasoning
Original note title
partial symbolic abstraction preserves information completeness that full formalization loses — augmentation outperforms replacement for logical reasoning