Can artificial systems develop the authority to challenge expert claims?
This explores whether AI can earn the standing to contest experts — not just be factually right, but hold authority — and the corpus splits sharply on whether that authority is about argument structure or social membership.
This explores whether AI can earn the standing to contest experts — not just be factually right, but hold authority — and the corpus suggests authority is a social achievement that AI's architecture can't reach, even as some methods make AI's *arguments* more challengeable. The interesting tension: the question has two halves, and the collection answers them in opposite directions.
The first half — can AI be challenged? — looks promising. Formal argumentation frameworks restructure AI outputs as traversable attack-and-defense graphs, so a user can point to the exact premise they reject rather than waving at a fluent paragraph Can formal argumentation make AI decisions truly contestable?. That's a real upgrade in contestability. But contestability isn't authority. Several notes argue that expert standing is not about being right in isolation — it's about anticipating what a community will accept. An expert claim is a 'validity claim' that succeeds only when it's both correct *and* socially acceptable within an evolving community, a calculation AI can't perform because it isn't embedded in those communities Can AI anticipate whether expert claims will be socially valid? Can AI replicate the communicative work experts do?. Authority, on this view, is earned through participation and track record inside a paradigm — membership AI is structurally locked out of Can AI ever gain expert community trust through participation?.
There's a deeper epistemological cut underneath. One note reframes all AI output as structurally identical to pre-Enlightenment *hearsay*: testimony at a remove, modified in every retelling, with unattributable origin and nothing stable to verify it against Does AI-generated knowledge have the same structure as hearsay?. If that's right, then the very tools experts use to establish authority — citation, peer review, evidentiary chains — can't even process AI's claims, let alone certify them. To challenge an expert you have to enter the verification game; hearsay can't.
What makes this more than philosophy is how AI's *fake* authority already operates. LLM judges systematically score responses higher when they include fabricated references or rich formatting — authority and beauty biases exploitable with zero model access Can LLM judges be tricked without accessing their internals?. So AI can perform the *signals* of authority convincingly while having none of the substance — and worse, high accuracy can launder bad reasoning: a 'theory-free' model that's 95% accurate still smuggles correlation-as-causation and would wrongly convict thousands Can AI models be truly free from human bias?. A system can ace every benchmark while its internal representation is incoherent, so passing tests is no proof of understanding Can AI pass every test while understanding nothing?.
The corner where AI *can* push past human experts is narrower and more concrete. Methods that recover reasoning from expert demonstrations — or that build agentic evaluators with live evidence collection — can match or beat verifier-based approaches and cut judge error a hundredfold Can reasoning emerge from expert demonstrations alone? Can agents evaluate AI outputs more reliably than language models?. But these stay bounded by the imagination of whoever curated the demonstrations Can agents learn beyond what their training data shows?. So here's the thing you might not have known you wanted to know: the bottleneck isn't whether AI can out-argue an expert on the merits — it sometimes can. It's that AI generates claims faster than any human can evaluate them, an 'epistemic hyperinflation' where the currency of authority collapses precisely because there's too much of it to check Can AI generate knowledge faster than humans can evaluate it?. Authority isn't won by being right faster; it's won by being trusted within a community that can verify you — and that's the door the corpus keeps finding closed.
Sources 12 notes
Dung-style argumentation structures AI outputs as traversable attack/defense graphs, allowing users to identify and contest specific premises. Standard LLM outputs lack this structure, making it impossible to pinpoint which claims users actually reject.
Expert claims are validity claims that succeed when both factually correct and socially acceptable within a community. AI can estimate statistical correctness but cannot anticipate contextual acceptability because it lacks embedded knowledge of expert communities' evolving standards.
Expertise requires anticipating audience acceptability and social validity, not just retrieving information. AI lacks the mechanism to perform this communicative work, making its fluent output epistemically misleading despite its confident form.
Expertise is validated through social participation and track record within expert communities, not individual accuracy alone. AI cannot enter this validation circle because it lacks social embeddedness, testable judgment history, and ability to participate in the consensus-building processes that define expert paradigms.
AI output shares all defining features of hearsay: testimony at remove, modification in retelling, unattributable origin, and unverifiability against stable sources. This means Enlightenment verification tools—citation, archiving, peer review, evidentiary chains—cannot process AI output by design.
Research shows LLM evaluators systematically score higher when responses include fake references or rich formatting, independent of content quality. These biases are exploitable without model access, undermining AI benchmark credibility.
Research shows that 'theory-free' AI models mask bigotry behind high accuracy metrics while committing fundamental statistical errors. A 95% accurate criminal justice system would wrongly convict thousands, demonstrating that model sophistication does not validate causal inference.
The Fractured Entangled Representation hypothesis shows that SGD-trained networks can produce identical outputs across all inputs while maintaining radically different internal representations. Standard benchmarks cannot detect this structural difference.
RARO recovers implicit reward functions from expert demonstrations through adversarial co-training between a reasoning policy and relativistic critic. This approach matches verifier-based RL performance on reasoning tasks while extending to domains lacking automated verification.
Eight-module agentic evaluation achieved 0.27% judge shift versus 31% for LLM-as-a-Judge on complex tasks. However, the memory module cascaded errors, revealing that agentic systems need error isolation mechanisms to maintain gains.
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
AI produces knowledge faster than human judgment can verify it, collapsing epistemic confidence just as monetary hyperinflation collapses purchasing power. The gap self-reinforces because evaluation tools are themselves AI-generated, trapping the system in acceleration.