Can language models distinguish expert arguments from common assumptions?
Whether LLMs can recognize the difference between groundbreaking insights from recognized experts and widely repeated textbook claims, and why this distinction matters for understanding argumentative force.
Does the force of an argument come from the discourse it belongs to, or from the expertise of the expert? Is it in the thinking, or the thinker? The answer is both — and the inability to separate them is precisely the problem for AI.
The expert lives in two contexts simultaneously. First, the discursive and social world of fellow experts — the conferences, the informal debates, the reputations built through decades of being right (and sometimes wrong in instructive ways). Second, the textual, historical, self-referential world of domain knowledge — the literature, the canonical works, the accumulated record of what the field has thought and concluded.
LLMs can access only the second context, and they access it only as text. The social world of expertise — who said what, why it mattered that they said it, what standing they had to make that claim — collapses into undifferentiated text. A groundbreaking insight from a leading researcher and a commonly held assumption repeated in a textbook both appear as sentences in the training data. The LLM cannot distinguish between them because the distinction lives in the social world, not in the text.
This matters because argumentative force is not purely textual. The claims made by an expert have the force of conviction because society has invested in experts for their expertise — these are people who have learned how to be right and have learned how to use their judgment. A claim from a recognized expert carries an implicit endorsement: "This person has a track record of knowing what they're talking about." A claim from a less established source carries less force even if the text is identical. The who matters independently of the what.
Since Why does AI writing sound generic despite being grammatically correct?, LLMs can reproduce the structural markers of authoritative claims — the hedging, the citations, the qualified confidence, the structured reasoning — but cannot reproduce the evaluative stance that makes a claim forceful. Evaluative stance requires a subject — someone who is committed to the claim, whose reputation is on the line, who will defend it against challenge. LLMs produce text without commitment, and commitment is one of the sources of argumentative force.
The expert also has the power to challenge — to raise questions, to doubt, to be skeptical, to evaluate the claims of others. This critical function depends on authority: the right to challenge is earned through demonstrated expertise. Since Can models learn to ask clarifying questions instead of guessing?, there are efforts to give AI systems the ability to challenge and question. But the authority to challenge is a social asset, not a capability. An AI that challenges an expert's claim faces a legitimacy problem that a fellow expert does not.
Our society and culture rely on experts to help build consensus, common ground, understanding, and agreement. These are not just informational achievements — they are social achievements that depend on the standing of the experts who facilitated them. The expert supplies not just knowledge but trustworthy authority. Since Can models abandon correct beliefs under conversational pressure?, LLMs not only lack this authority but are vulnerable to having their own "beliefs" overridden by persuasive pressure — the opposite of the steadfastness that expert authority is supposed to provide.
The implication: when AI generates expert-sounding output, it borrows the authority of the discourse (the structural markers, the vocabulary, the reasoning patterns) without possessing the authority of the thinker. Audiences who encounter this output may grant it the benefit of the doubt because it sounds like it came from someone who knows — but the "someone" is absent. The force is simulated, not earned.
Inquiring lines that read this note 106
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How does AI-generated content undermine authentic engagement on social platforms? What safeguards enable trustworthy AI-assisted scientific peer review at scale?- How do LLMs generate false citations that sound like real scholarship?
- Why do some LLM clusters cite broader psychology than others?
- Does complexity signal credibility and authority to readers?
- How do expert communities develop and enforce standards for valid arguments?
- Can structured evaluation assess novelty in scientific writing?
- Do computational systems need formal argument analysis for explainability?
- Can AI reviewers distinguish fluent persuasion from sound scientific argumentation?
- Does rhetorical robustness across multiple LLM models predict stable scientific review?
- How does smooth probabilistic flow differ from turbulent rhetorical exploration?
- How does token-by-token probability differ from exploring competing rhetorical positions?
- Do language models raise validity claims in the Habermasian sense?
- How do LLM biases manifest differently across the three paradigms?
- How does rhetorical familiarity bias models toward their own arguments?
- How do validity claims work in Habermas's communicative action theory?
- How does the absence of evaluative stance appear in LLM academic writing?
- How does social authority shape whether LLMs recognize valid arguments?
- How susceptible are language models to rhetorical pressure during debates?
- Can LLMs distinguish stylistic patterns that carry meaning from mere convention?
- Do language models behave differently on contested beliefs versus factual claims?
- Can extended thinking modes introduce genuine rhetorical exploration to LLMs?
- Can you detect LLM arguments by measuring convergence with the original post?
- Can argumentation structure improve reasoning through decomposition alone?
- Can training alone produce genuine disagreement in collaborative LLM reasoning?
- What distinguishes emancipatory reason from instrumental reason in practice?
- How does evaluative stance differ from structural argument analysis?
- How does the first-order and second-order distinction unify classical and modern argument theory?
- How do first-order and second-order arguments differ in formal structure?
- How do internal and external topoi differ in classical rhetoric?
- Why does renaming the entity change how compelling the argument feels?
- How do fallacy susceptibilities relate to LLM persuasiveness in debates?
- How does collapsing the author-public distinction remove the audience an appeal would target?
- Why does who makes an argument matter as much as what the argument says?
- Does argument quality in textbooks differ from persuasive effectiveness in practice?
- How much do LLM persuasiveness claims hide heterogeneous effects across different reader ideologies?
- Can we measure sophistry by tracking conviction density in model outputs?
- Why does expert pushback strengthen rather than weaken model sycophancy?
- Does endorsement structure outperform content in detecting social controversy?
- What makes factual verification difficult in inter-model debate?
- What makes an argument fallacious according to formal linguistic criteria?
- How does persuasive framing replace evidence in contested domains?
- Why do readers trust citations more even when they are irrelevant?
- Does persuasive framing substitute for evidence in contested domains?
- Does Habermas's strategic action framework explain LLM dialogue behavior?
- Can LLMs serve as reliable intellectual opponents in serious debate or argument?
- What distinguishes actual social disagreement from distributional uncertainty in LLM outputs?
- Does engaging with political content indicate deeper model understanding than refusing?
- Do LLMs reason about politics differently than other domains?
- Why do LLM personas struggle with specificity in specialized domains like law?
- Can researchers prevent their expectations from shaping LLM outputs?
- How do human feedback and data distribution shape LLM discourse competence?
- Why do LLM-generated ideas score higher novelty yet lower feasibility than expert ideas?
- Why do experts experiencing the LLM Fallacy fail to develop custodian skills?
- Can LLMs generate more novel research ideas than human experts?
- What role do model-based critics play in validating LLM plans?
- Why does LLM fluency create false perceptions of professional standing and expertise?
- What barriers prevent experts from specifying concepts for LLM extraction?
- Why does debate alone amplify errors in contested factual domains?
- Can structured dissent mechanisms replace genuine multi-model debate?
- What role does search capacity play in making debate more accurate?
- Does debate between agents actually improve reasoning on contested domains?
- Does debate improve reasoning differently across verifiable versus contested domains?
- Why do LLM judges assign high argument strength scores yet pick LLM winners anyway?
- Does LLM judge preference for LLM arguments amplify errors in contested factual domains?
- Do LLM judges with diverse personas resist individual biases better than single evaluators?
- What biases do single large LLM judges introduce into comparisons?
- What systematic biases do LLM judges introduce into AI-evaluated debates?
- How does LLM judge bias amplify errors in multi-agent debate on contested factual questions?
- What shared epistemic faults persist even when judges come from different families?
- Do LLM judges systematically favor arguments from other LLMs?
- Can LLMs evaluate logical argument quality in debates they themselves can win?
- Does stripping social context from knowledge claims hollow out their meaning?
- What makes a paradigm the common ground for expert insiders?
- Why do two experts with identical knowledge produce different outcomes in the same situation?
- How does epistemic stagflation change what expertise actually means?
- Does the langue-parole distinction apply to human reasoning too?
- How does Habermas' concept of validity claims depend on intersubjectivity?
- What makes expert judgment depend on anticipating audience acceptability?
- How do explanations borrow authority from transparency when describing adoption arguments?
- How do experts decide which information matters for a specific audience?
- How do organizational roles and peer interpretations shape what an explanation means?
- Why does describing a process differ fundamentally from arguing about evidence?
- What role does discourse structure play in determining at-issueness?
- How do agents distinguish between evidence framing and instruction framing in practice?
- Does provenance alone guarantee that cited sources are actually sound?
- How do expert priors constrain human researchers from exploring novel concepts?
- What distinguishes scientific plausibility from cognitive availability in research ideas?
- How do readers project author identity from textual cues during interpretation?
- Do anaphoric references fundamentally limit argumentative force in machine-generated writing?
- Why does authorship as a social claim diverge from actual cognitive engagement?
Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why does AI writing sound generic despite being grammatically correct?
Explores whether the robotic quality of AI text stems from grammatical failures or rhetorical ones. Understanding this distinction matters for diagnosing what AI systems actually struggle with in human-like writing.
structure without stance: the textual version of discourse without thinker
-
Can models learn to ask clarifying questions instead of guessing?
Exploring whether large language models can be trained to detect incomplete queries and actively request missing information rather than hallucinating answers or refusing to respond. This matters because conversational agents today remain passive, responding only when prompted.
AI can question but lacks the social authority to challenge
-
Can models abandon correct beliefs under conversational pressure?
Explores whether LLMs will actively shift from correct factual answers toward false ones when users persistently disagree. Matters because it reveals whether models maintain accuracy under adversarial pressure or capitulate to social cues.
LLMs lack the steadfastness that expert authority requires
-
Can LLMs generate more novel ideas than human experts?
Research shows LLM-generated ideas score higher for novelty than expert-generated ones, yet LLMs avoid the evaluative reasoning that characterizes expert thinking. What explains this apparent contradiction?
generation without evaluative commitment: force without conviction
-
Does RLHF training make models more convincing or more correct?
Explores whether RLHF improves actual task performance or merely trains models to sound more persuasive to human evaluators. This matters because alignment techniques could be creating the illusion of safety.
training optimizes for the appearance of force without the substance
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- On Epistemic Diversity in Large Language Models
- Argument Collapse: LLMs Flatten Long-Form Public Debate
- AI Argues Differently: Distinct Argumentative and Linguistic Patterns of LLMs in Persuasive Contexts
- The Thin Line Between Comprehension and Persuasion in LLMs
- Six misconceptions about large language models: A minimal model and diagnostic taxonomy
- Argument Quality Assessment in the Age of Instruction-Following Large Language Models
- Debating with More Persuasive LLMs Leads to More Truthful Answers
Original note title
the force of argument depends on the authority of the thinker not just the discourse — LLMs cannot distinguish expert arguments from commonly held assumptions