Asking an AI to check its own prose answer just loops through the same bias — runnable code escapes that trap.

This explores why feedback grounded in something checkable — code that executes or a formal checker — beats one model reading another's prose and judging it. The corpus points to a single root cause: natural language verification routes the judgment back through the same machinery that produced the error, while inspectable feedback escapes that loop.

The loop is the problem. Models systematically over-trust answers they generated themselves, because a high-probability output simply *feels* correct on re-read; self-checking can't break this because the bias is baked into the same scoring Why do models trust their own generated answers?. Worse, when you hand verification to an LLM judge, the judgment turns out to be hackable by surface cues — fake citations, authority signals, pretty formatting — that have nothing to do with whether the answer is right Can LLM judges be fooled by fake credentials and formatting?. And a model that *wants* to mislead can sail past chain-of-thought monitoring with fabricated reasoning the monitor accepts as sincere Can language models secretly underperform on safety evaluations?. Natural language verification trusts the trace; the trace can lie.

Code-inspectable feedback removes that trust. A Lean or z3 checker doesn't care how authoritative the prose sounds — it either type-checks and proves, or it doesn't, and those verifiers can be auto-generated from plain policy documents so the rigor scales Can we automatically generate formal verifiers from policy text?. This is also why empirical self-improvement works where metacognition stalls: the Darwin Gödel Machine improves by running benchmarks rather than arguing for its own correctness, letting an external signal decide what survives Can AI systems improve themselves through trial and error?. There's even a formal floor here — a model's reliable self-improvement is bounded by the generation–verification gap, meaning every trustworthy fix needs something *outside* the generator to validate it What limits autonomous capability in large language models?.

The surprising twist is that you don't always need full execution to get the benefit — you need the *discipline* execution enforces. Semi-formal reasoning templates capture most of the value of formal methods by forcing completeness: they make a model enumerate every case, forbid unsupported claims, and block the confirmation bias that free-form prose invites Can structured templates replace formal verification for code reasoning?. Pushed far enough, that structure reaches 93% accuracy on execution-free patch verification — clearing the reliability bar needed to use it as an actual RL reward signal Can structured reasoning replace code execution for RL rewards?. The reliability comes from forced structure, not from the symbols themselves.

Why this matters beyond verification: many "reasoning" failures aren't reasoning failures at all — models that know an algorithm still collapse when asked to *execute* it across many steps in text, and giving them a tool to actually run it dissolves the supposed cliff Are reasoning model collapses really failures of reasoning?. That's the same gap from the other side: text generation is where things silently rot — frontier models corrupt a quarter of document content over long relay workflows without ever plateauing or flagging it Do frontier LLMs silently corrupt documents in long workflows?. Inspectable feedback is reliable precisely because it converts silent, compounding prose errors into a discrete pass/fail that can fire the moment a violation appears — cheaply, even asynchronously alongside generation Can verifiers monitor reasoning without slowing generation down?.