Can autoformalisation from natural language preserve semantic accuracy?
This explores whether translating natural language (math, specs, prose) into formal, machine-checkable logic can keep the original meaning intact — and what in the corpus tells us when that translation drifts.
This explores whether translating natural language into formal logic preserves what the source actually meant — the autoformalisation problem. The corpus doesn't have a paper on autoformalisation by name, but it has a sharp set of results on the exact bottleneck autoformalisation runs into: language models operate on form, not meaning, and the gap between the two is where semantic accuracy leaks out.
The deepest version of the worry is foundational. Bender & Koller's argument is that meaning lives in the relation between expressions and communicative intent, and a model trained only on form-to-form prediction has no access to that relation Can language models learn meaning from text patterns alone?. Autoformalisation is precisely a meaning-preservation task — you want the formula to mean what the sentence meant — so if models track surface patterns rather than meaning, the translation is structurally at risk. That risk shows up concretely: models systematically prefer higher-frequency surface phrasings over semantically equivalent rare ones Do language models really understand meaning or just surface frequency?, which is bad news for formalising unusual or precisely-worded statements where the rare phrasing is the whole point.
It gets worse where formalisation gets hardest. Autoformalisation requires parsing nested quantifiers, embedded clauses, and compound noun phrases — and that's exactly the structure on which LLMs degrade predictably. Top-tier models consistently misidentify embedded clauses and complex nominals, with accuracy falling as syntactic depth increases Why do large language models fail at complex linguistic tasks?. A long or deeply-nested specification compounds the problem, since reasoning quality drops with input length well before any context limit Does reasoning ability actually degrade with longer inputs?. And when the source statement conflicts with what the model 'expects' to be true, its training priors can override the text in front of it Why do language models ignore information in their context? — so a counterintuitive premise may get quietly formalised into the conventional one.
There are two reasons not to despair, and they're the more interesting half of the picture. First, formalisation is unusually *checkable*. The corpus's most useful idea for autoformalisation isn't about translation at all — it's the generation-verification gap: models can't reliably improve themselves, but they can be pinned by an external verifier What stops large language models from improving themselves?. Formal targets (proof assistants, type checkers) are exactly such verifiers. The catch is subtle: a verifier confirms the formula is *valid*, not that it *matches the source* — so verification catches malformed output but not faithful-looking mistranslation. Second, models can do genuine structural analysis when forced to reason explicitly: o1 constructs syntactic trees and phonological generalisations step by step Can language models actually analyze language structure?, suggesting that the metalinguistic competence autoformalisation needs is reachable through reasoning rather than one-shot pattern matching.
The synthesis the corpus points to: semantic accuracy is preservable, but not for free and not by the model alone. Naive one-shot formalisation inherits every form-over-meaning failure above; what closes the gap is the pairing of explicit reasoning with an external check — and even then you need to verify *fidelity to the source*, not just internal validity, since that's the one thing a proof checker won't tell you. Worth knowing too: long-context models can absorb a spec but still stumble on tasks needing exact relational structure Can long-context LLMs replace retrieval-augmented generation systems? — a reminder that 'reading the whole document' and 'getting the structure right' are different skills, which is the autoformalisation problem in miniature.
Sources 8 notes
Bender & Koller argue that meaning requires the relation between expressions and communicative intents. Since LLMs are trained only on form-to-form prediction with no access to shared attention or intent, they cannot reconstruct the meaning that grounds language.
LLMs show consistent preference for higher-frequency surface forms over semantically equivalent rare paraphrases across math, machine translation, commonsense reasoning, and tool calling. This suggests models track statistical mass from pretraining rather than meaning-recognition as their primary mechanism.
Top-tier LLMs like Llama3-70b consistently misidentify embedded clauses, verb phrases, and complex nominals. Performance degrades predictably as syntactic depth increases, revealing that statistical learning captures surface patterns but not deep grammatical rules.
FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
OpenAI's o1 model successfully constructs syntactic trees and phonological generalizations through explicit step-by-step reasoning, revealing that LLM linguistic capability extends far beyond behavioral language tasks to genuine language analysis.
The LOFT benchmark shows LCLMs match RAG on semantic retrieval without explicit training, but cannot execute relational queries requiring joins across structured tables. Context length alone cannot bridge this gap.