Can argumentation structure improve reasoning through decomposition alone?
This explores whether breaking reasoning into argument components — premises, warrants, attack/defense moves — improves how LLMs reason on its own, or whether the structure only helps when paired with something else.
This explores whether argumentation structure improves reasoning *through decomposition alone* — that is, whether the act of breaking reasoning into explicit argument parts is what does the work, or whether structure is doing something subtler. The corpus suggests the answer is a qualified yes, with a sharp caveat: decomposition helps most when it forces a model to check something it would otherwise skip, and helps least when it's just rearranging the same output into a tidier shape.
The strongest evidence for decomposition itself paying off comes from structured prompting that makes implicit steps explicit. Applying Toulmin's model as prompting steps (CQoT) improves reasoning specifically by forcing models to surface warrants and backing rather than gliding over the unstated premise — it catches failures that plain chain-of-thought lets through Can structured argument prompts make LLM reasoning more rigorous?. Structuring a single model's reasoning as a dialogue between distinct agents, rather than one monologue, similarly improves diversity and coherence by breaking the fixed-strategy rut that monologue reasoning falls into Can dialogue format help models reason more diversely?. And formal argumentation frameworks turn an answer into a traversable graph of attacks and defenses, letting a user pinpoint and contest a specific premise — something an unstructured paragraph can't offer Can formal argumentation make AI decisions truly contestable?. In each case the gain comes from the decomposition compelling a check or a perspective the model wouldn't have produced flat.
But a counter-current in the corpus warns against crediting structure alone. Chain-of-thought reasoning looks like decomposition, yet it's better described as constrained imitation: models reproduce the *form* of reasoning by pattern matching, which is why format effects dominate content and why structurally invalid prompts can still succeed What makes chain-of-thought reasoning actually work?. If form can succeed while being invalid, then the shape of decomposition isn't guaranteeing the inference. Reinforcing this, Chain of Draft matches verbose chain-of-thought at 7.6% of the tokens — meaning the vast majority of a 'decomposed' explanation was style and documentation, not computation Can minimal reasoning chains match full explanations?. Verbosity is even a steerable linear direction in activation space, separable from accuracy Can we steer reasoning toward brevity without retraining?. So adding more visible structure is not the same as adding more reasoning.
There's also a hard ceiling on how far decomposition can be pushed. Classifying argument *schemes* — recognizing inferential patterns across distributed text — carries a higher cognitive load than tagging components or stance, and models plateau at F1 0.55–0.65 even where they top 0.80 on the easier sub-tasks Why does argument scheme classification stumble where other NLP tasks succeed?, with reliable scheme classification only emerging few-shot in larger models Can large language models classify argument schemes reliably?. Decomposition assumes a clean target to decompose toward, but argument reconstruction is fundamentally underdetermined: multiple valid formalizations of the same text exist with no ground truth Why do different people reconstruct the same argument differently?. And some apparent reasoning collapses aren't reasoning failures at all — they're execution failures, where a model knows the algorithm but can't carry out the steps at scale Are reasoning model collapses really failures of reasoning?. Decomposing a problem the model can't *execute* won't rescue it.
The synthesis, then: argumentation structure improves reasoning not by decomposition as such, but by decomposition that imposes a *constraint* — a warrant that must be named, an attack that must be answered, a counter-perspective that must be voiced. Where the structure is merely descriptive (longer traces, prettier formatting), it adds tokens, not thought. The interesting frontier the corpus points to is that argument structure's real payoff may be less about better answers and more about *contestability* — making reasoning something a person can inspect and push back on Can language models distinguish expert arguments from common assumptions? — which is a different and arguably more valuable win than raw accuracy.
Sources 11 notes
Applying Toulmin's argument model as explicit prompting steps (CQoT) improves LLM reasoning by forcing models to identify warrants and backing rather than skipping implicit premises. The method catches failures that standard chain-of-thought prompting allows.
DialogueReason, which structures a single model's internal reasoning as dialogue between distinct agents in separate scenes, overcomes monologue reasoning's fixed-strategy and fragmented-attention weaknesses, especially on tasks requiring multiple problem-solving approaches.
Dung-style argumentation structures AI outputs as traversable attack/defense graphs, allowing users to identify and contest specific premises. Standard LLM outputs lack this structure, making it impossible to pinpoint which claims users actually reject.
CoT systems reproduce the form of reasoning through pattern matching rather than performing genuine logical inference. This explains why format effects dominate content, why structurally invalid prompts succeed, and why stronger reasoning models become less instruction-compliant.
Chain of Draft achieves equivalent accuracy to standard chain-of-thought on arithmetic, symbolic, and commonsense tasks while using only 7.6% of tokens. The 92.4% of removed tokens served style and documentation, not computation.
Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.
Scheme classification requires recognizing inferential patterns across distributed text spans, not local surface features. Models plateau at F1 0.55–0.65 while the same systems exceed 0.80 on component tagging and stance, suggesting the integrative reasoning demand is fundamentally different.
Zero-shot prompting fails uniformly across models. Few-shot with scheme descriptions helps, but only larger models exceed F1 0.55, with Claude reaching 0.65. Smaller models plateau around 0.53, suggesting a representational capacity threshold.
Multiple valid argument reconstructions exist for the same text with no ground truth. This is not annotation error but an inherent feature of the task—different formalization schemas are each internally valid.
Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.
LLMs lose the social context that gives expert claims their force—reputation, track record, and standing—because they process only text, not the social world where expertise is built and evaluated.