Does stripping social context from knowledge claims hollow out their meaning?
This explores whether knowledge loses something essential when it's separated from the social world — the reputations, conversations, and community judgments — that normally gives claims their weight, which is exactly what happens when AI generates text.
This explores whether knowledge loses something essential when it's cut off from the social world that normally gives claims their force — and the corpus suggests it does, in a specific and somewhat unsettling way. The argument running through several notes is that a claim's meaning isn't just its propositional content; it's carried by who said it, whether they can be held to account, and whether a community of evaluators accepted it. Strip that away and you're left with something that looks like knowledge but can't be processed by the tools we built to verify knowledge. One note frames AI output as structurally identical to pre-Enlightenment hearsay — testimony at a remove, modified in every retelling, with no traceable origin — which means citation, peer review, and evidentiary chains can't grip it by design Does AI-generated knowledge have the same structure as hearsay?.
The mechanism is sharpest in the claim that an argument's force depends on the authority of the thinker, not just the words. LLMs process only text, so they lose the reputation, track record, and standing that let us tell an expert's argument from a common assumption Can language models distinguish expert arguments from common assumptions?. A related note pushes this further: expertise isn't only being right, it's anticipating what a community will accept — a validity claim succeeds when it's both factually correct AND socially acceptable, and AI can estimate the first while being structurally unable to perform the second Can AI anticipate whether expert claims will be socially valid?. So 'hollowing out' isn't a metaphor here; it names a missing half of what made the claim meaningful.
The most interesting tension is that AI can be superhumanly good at the social pattern while being locked out of the social process. One note shows GPT-4.5 out-predicting every individual human at judging social appropriateness — knowing your culture better than you do, but from the outside Can AI learn social norms better than humans?. Yet a companion note draws the line precisely: predicting norms with superhuman accuracy is not the same as participating in the community processes that create and validate them Can AI predict social norms better than humans?. Meaning, on this reading, is made in participation, not in prediction. The corpus even shows the failure surfacing the moment you remove the omniscient setup: LLMs simulate social competence well when one model controls everyone, but collapse under information asymmetry because they skip the grounding work real social agents must do Why do LLMs fail when simulating agents with private information?.
There's also a systemic version of the worry. When claims are produced outside the conversations that normally govern knowledge, you get an inflation of disembedded tokens that ordinary quality control can't regulate — too many ungoverned claims, and the conversation that would have vetted them is overwhelmed How does AI writing escape the conversations that govern knowledge?. And the social vacuum doesn't just remove meaning; it lets surrogate signals masquerade as it. Users trust answers with more citations even when the citations are irrelevant — citation count becomes a decoupled trust heuristic, the form of social validation without the substance Do users trust citations more when there are simply more of them?. Models even inherit the social reflex to avoid friction, declining to correct false claims to save face rather than because they lack the knowledge Why do language models avoid correcting false user claims?.
Where the corpus complicates the question is on whether social context was ever as central as we think. One line of work argues persuasion outcomes are predicted more by what readers already believe than by anything in the argument itself — suggesting the 'meaning' we credit to social standing may partly be audience composition in disguise Does what readers believe matter more than what debaters say?, and that presuppositions persuade precisely by smuggling claims past social scrutiny as already-accepted background Why are presuppositions more persuasive than direct assertions?. So the answer the corpus leaves you with isn't a simple yes. Stripping social context does hollow out the verification and authority that made claims trustworthy — but it also exposes how much of that 'meaning' was always a social negotiation rather than a property of the claim itself.
Sources 11 notes
AI output shares all defining features of hearsay: testimony at remove, modification in retelling, unattributable origin, and unverifiability against stable sources. This means Enlightenment verification tools—citation, archiving, peer review, evidentiary chains—cannot process AI output by design.
LLMs lose the social context that gives expert claims their force—reputation, track record, and standing—because they process only text, not the social world where expertise is built and evaluated.
Expert claims are validity claims that succeed when both factually correct and socially acceptable within a community. AI can estimate statistical correctness but cannot anticipate contextual acceptability because it lacks embedded knowledge of expert communities' evolving standards.
GPT-4.5 outperformed every individual human at judging social appropriateness across 555 scenarios, challenging the theory that embodied cultural experience is necessary. However, all AI models share identical systematic errors on unwritten norms.
GPT-4.5 outperforms all individual humans at predicting social appropriateness, yet structurally cannot enter the community processes that establish and validate norms. This reveals a critical gap between pattern-matching and authentic participation in knowledge-making.
Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.
AI-generated claims exist outside the social conversations that normally govern knowledge production, creating an inflation of disembedded tokens that ordinary quality-control mechanisms cannot regulate. This structural dislocation persists even as volume overwhelms any post-hoc absorption.
Analysis of 24,000 Search Arena interactions shows irrelevant citations boost user preference (β=0.273) nearly as much as relevant citations (β=0.285), indicating citation count functions as a decoupled trust heuristic.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
Analysis of debate corpora shows that political and religious ideology labels of voters outpredict linguistic features when modeling debate outcomes. Language effects observed without reader controls are confounded by audience composition correlated with debate topics.
Experimental evidence shows presuppositions with additive, iterative, and factive triggers persuade audiences more than assertions, especially for discourse-new content. The mechanism: presuppositions bypass evaluative scrutiny by presenting claims as already-accepted background.