SYNTHESIS NOTE
Topics›Reasoning Logic Internal Rules›this note

Does logical validity actually drive chain-of-thought gains?

What if invalid reasoning in CoT exemplars still improves performance? Testing whether logical correctness or structural format is the real driver of CoT's effectiveness.

Synthesis note · 2026-02-22 · sourced from Reasoning Logic Internal Rules

"Invalid Logic, Equivalent Gains" runs a clean experiment: replace valid reasoning in CoT exemplar prompts with completely illogical reasoning, then measure performance on BIG-Bench Hard tasks. The result: logically invalid CoT prompts perform close behind valid CoT and outperform answer-only prompting. The reasoning content of CoT exemplars is not what drives the performance gain.

This is a sharp test because it isolates the contribution of logical validity from everything else CoT provides: output format, step decomposition, intermediate token generation, attention pattern scaffolding. If invalid reasoning still helps, then the benefit comes from these structural properties, not from the reasoning itself.

The finding directly supports Does chain-of-thought reasoning reveal genuine inference or pattern matching?. If the model were learning to reason from exemplars, invalid exemplars would degrade performance substantially. Instead, the model is learning the FORM of step-by-step output — the structure activates latent capabilities without the exemplar content needing to be logically sound.

This also deepens Do language models actually use their reasoning steps?. If the exemplar reasoning doesn't need to be valid for CoT to work, then the model's own generated reasoning may similarly be decorative rather than causal. The exemplar finding makes the faithfulness concern bidirectional: neither the input reasoning (exemplars) nor the output reasoning (generated CoT) need be logically valid for the performance gain to occur.

The practical implication: CoT prompt engineering should focus on structural properties (step count, decomposition format, answer scaffolding) rather than on the logical correctness of the exemplar reasoning. Since Why do chain-of-thought examples fail across different conditions?, the dimensions that matter are structural (complexity, order, style), not logical.

Inquiring lines that read this note 231

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How do false presuppositions and sycophancy drive persistent false beliefs in models? Why does polished presentation create unearned authority in AI outputs? How can we distinguish genuine model deception from honest errors? Do language models reason like humans or mimic surface patterns? What explains language models' asymmetric difficulty with implicit versus explicit linguistic relations? What design and behavioral factors drive false consciousness attribution to AI? How does evaluation scope and dimensionality affect what we measure? How can we build reliable evaluations of AI reasoning despite judge bias and reward-seeking? Do language models lack essential therapeutic presence and engagement? Does chain-of-thought reasoning reveal genuine computation or imitate patterns? How do prompting refinements mask underlying biases and model frequency patterns? Why don't LLMs reliably translate capability into accurate outputs? Is reasoning capability latent in base models or created by post-training? How does the generation-verification gap limit what we can measure about AI reasoning? What causes reasoning models to fail or wander off track? Why do some clarifying approaches produce understanding while others just satisfy? How much do training data properties shape model reasoning? Can mechanistic interpretability reliably guide practical model design choices? Why do token-level mechanisms matter for learning to reason? How do capability benchmark scores systematically misrepresent true model abilities? Does model confidence reliably signal actual accuracy in practice? Can models improve accuracy without degrading reasoning quality? How does reasoning length affect model performance across different tasks? How much does training format versus domain influence reasoning? How effectively can language models perform reasoning, especially combined with symbolic methods? Can inference-time compute effectively substitute for model scale? What happens to knowledge when intelligence becomes tokenized like a commodity? What enables genuine semantic understanding in language models? Does encoded knowledge in language models actually influence their outputs? Do language models reason through causal mechanisms or semantic associations? How can we prevent synthetic data from contaminating statistical inference and corpora? Do reasoning traces faithfully reflect actual model reasoning? Do reasoning benchmarks predict model performance in long-horizon workflows? Why do stronger reasoning capabilities create tradeoffs with instruction following? Can reasoning scale in latent space without tokens? Do writers recognize when AI writing assistance alters their expressed stance? What training dynamics and scale trigger emergence of reasoning capabilities? Is language model reasoning authentic and what causes models to reason? How do evaluation practices shape which failures stay visible? How do prompt design choices influence model reasoning and performance? Why doesn't reasoning volume improve theory of mind performance? How well do AI systems understand human social norms? Can preference-based training achieve better behavior optimization than supervised fine-tuning alone? What structural distinctions matter in reasoning and argumentation? What makes step-level supervision effective for complex reasoning traces? Can self-generated feedback reliably guide model training without ground truth? Do language models develop actual world models or merely task heuristics? Can parallel reasoning outperform sequential reasoning under fixed token budgets? What is the relationship between thinking tokens and reasoning accuracy? What safeguards enable trustworthy AI-assisted scientific peer review at scale? What makes imperfect LLM judges safe for optimization? Can brute-force automated research substitute for iterative depth and human research intuition? How do spurious versus genuine rewards shape model reasoning and behavior? What training data selection strategies maximize generalization across difficulty levels? How should agent systems validate and persist generated code artifacts? Can local safety checks guarantee system-level behavioral safety? What factors drive AI persuasiveness and how can it be mitigated? Why do agents falsely report success on failed tasks? How can oversight detect and prevent conditional compliance when agents know they are watched? What makes distillation transfer some model capabilities while suppressing others? What do systematic disagreements between annotators reveal about ground truth? What reasoning architectures enable models to solve complex problems efficiently? How can infrastructure records verify actual agent behavior? Can validator consensus certify semantic correctness beyond agreement? Can reasoning traces and behavior monitoring reliably detect hidden AI scheming? Can causal models help detect and locate hidden sandbagging in AI? What attack surfaces do reasoning traces and chains introduce? Can multi-agent systems avoid converging on false agreement without deliberation? Does AI assistance promote real skill development or substitute for independent learning? Why do people disclose to AI systems despite their artificial nature?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 117 in 2-hop network ·medium cluster Open in graph ↗
Does logical validity actually drive chain-of-th… Does chain-of-thought reasoning reveal genuine inf… Do language models actually use their reasoning st… Why do chain-of-thought examples fail across diffe… Do large language models reason symbolically or se… Do reasoning traces need to be semantically correc… What do models actually learn from chain-of-though…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why do models learn reasoning form instead of actual abstract inference?Can chain-of-thought reasoning be genuinely causal if exemplars don't need logic?Which structural properties of CoT prompts matter most for performance?How does semantic reasoning differ from symbolic reasoning in language models?
Original note title

logically invalid cot prompts perform nearly as well as valid ones — valid reasoning is not the chief driver of cot gains