Why do readers interpret the same sentence so differently?
How much of annotation disagreement in NLP reflects genuine interpretive multiplicity rather than error? This explores whether social position and moral framing systematically generate competing but equally valid readings.
The standard assumption underlying NLP benchmark design is that sentences have one correct interpretation. Disagreement between annotators signals annotation failure. The solution is to filter or adjudicate until one answer emerges.
Interpretation Modeling (IM, Cercas Curry et al. 2023) challenges this assumption directly. The study models multiple interpretations of socially embedded sentences, guided by reader attitudes toward the author and reader understanding of implicit moral judgments. Finding: conflicting interpretations are socially plausible. They reflect different social positions and moral framings, not annotation error.
This is not about ambiguous sentences in the traditional sense (lexical or syntactic ambiguity) but about the social and implicit dimensions of meaning in natural communication. A sentence embedded in a social context carries different meanings for readers with different:
- Relationships to the speaker
- Moral frameworks for evaluating the content
- Common ground with the speaker's implied community
The interpretations that result are not all "correct" in a truth-conditional sense, but they are all "valid" in a socially and pragmatically grounded sense — readers with different social positions genuinely understand different things from the same text.
The implication is uncomfortable for NLP: the gold standard that benchmarks aspire to may not exist for a substantial portion of natural language. Treating disagreement as noise produces evaluation systems that measure agreement on easy cases while missing the hard question of how interpretation actually works.
The NLI disagreement literature provides statistical confirmation. "Lost in Inference" (analyzing NLI annotation disagreement across major benchmarks) finds that NLI task performance is not saturated — humans continue to disagree, and that disagreement is not random noise but structured. Human annotation distributions on contested examples carry information that the majority label discards. This is the empirical grounding for IM's theoretical claim: interpretation is irreducibly multiple, and the distribution over interpretations is itself meaningful data.
An additional mechanism: social identity projection. Readers don't just apply their moral frameworks abstractly — they project the likely social identity of the author based on textual cues, then interpret the content through the lens of that projected identity. Two readers who project different author identities from the same text will read the same words as carrying different social stances. This is a grounding claim about interpretation that goes beyond semantic ambiguity.
This connects to Why do speakers deliberately use ambiguous language? — interpretive multiplicity is not a failure of specification but a feature of how socially embedded language operates. Since Do standard NLP benchmarks hide LLM ambiguity failures?, this irreducibility is doubly hidden.
Inquiring lines that use this note as a source 58
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- How does token-by-token probability differ from exploring competing rhetorical positions?
- Why does renaming the entity change how compelling the argument feels?
- How do readers selectively hold frame-related words in mind?
- Why does training data saliency distort how models judge meaning?
- Why does debate alone amplify errors in contested factual domains?
- What makes ambiguity recognition fundamentally important for poetry analysis?
- Why do stakeholders interpret the same explanation differently in practice?
- How do politeness strategies depend on semantic ambiguity between literal and intended meaning?
- What measurement artifacts emerge when annotators interpret the same question differently?
- Why do non-attitudes cluster around value-laden questions most relevant to alignment?
- How does semantic ambiguity differ from structural ambiguity in language?
- Is interpretive multiplicity a bug in language or a feature?
- What signals beyond surface content indicate a passage caused a user's reaction?
- What metrics actually measure disagreement in multi-turn conversations?
- Does endorsement structure outperform content in detecting social controversy?
- Can decreased engagement be distinguished from genuine semantic contradiction?
- How does fluent text output trigger misleading cognitive attributions in readers?
- How do organizational roles and peer interpretations shape what an explanation means?
- What semantic classifier design avoids lexical variation without genuine conceptual distinctness?
- Why does lexical difference fail to trigger reader suspicion of artificial origin?
- Can discourse communities collectively detect disruptions individual readers miss?
- What would it take for readers to inspect rather than assume authorship?
- How do humans detect which words belong to the same frame together?
- Can persona-based approaches capture genuine disagreement in expert annotations?
- How can structurally different text produce equivalent real-world effects?
- How do readers interpret AI text differently from human text?
- What distinguishes pseudo-objectivity from genuine intersubjective discourse?
- How does the inability to manage ambiguity undermine literary analysis tasks?
- Why does explanation source matter more than explanation content?
- How does frame selection differ from frame application in meaning-making?
- Why do different readers extract different meanings from identical text?
- How do cultural norms reshape initial interpretations of social intent?
- Why do NLP models fail at recognizing multiple valid interpretations?
- How do human annotators disagree systematically on ambiguous examples?
- Does removing information about who wrote something change how we interpret it?
- Does adding multiple interpretations to ambiguous situations respect language more than resolving them?
- Why do posters acknowledge multiple viewpoints without integrating them into coherent judgments?
- Do high-disagreement items signal contested values or measurement noise?
- Can the same predicate generate different projection strength in different contexts?
- Do chain-of-thought prompts help RLVR models predict annotation disagreement?
- Why do NLP benchmarks treat annotation disagreement as noise rather than signal?
- Can moral frameworks alone explain why readers understand sentences differently?
- How do readers project author identity from textual cues during interpretation?
- What information is lost when majority labels discard minority interpretations?
- How does alignment training suppress the kind of critical stance style interpretation needs?
- How do social position and moral framing create irreducibly different interpretations of reviews?
- How does unilateral interpretation differ from mutual communicative uptake?
- What makes some interpretive postures stick while others fail to form?
- How does semantic framing differ from content injection attacks?
- Why do high-disagreement tasks benefit from broad rater pools over deep annotation?
- How do annotation artifacts get mistaken for genuine human values?
- Can detectors trained for one task reliably perform differently on unexpected text sources?
- Do readers with weakly held priors respond more to linguistic features than ideologically committed ones?
- How much does reader ideology matter compared to the words being used?
- How does constitutional alignment compare to RLHF in removing human annotation costs?
- Why does fairness depend on context and who you ask?
- Can readers detect meaning through resonance patterns alone without knowing authorial intent?
- Where does the meaning actually originate in reader-detected resonance across language?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do speakers deliberately use ambiguous language?
Explores whether ambiguity is a linguistic defect or a strategic tool speakers use for efficiency, politeness, and deniability. Matters because it challenges how we train language systems.
interpretive multiplicity is functionally analogous to ambiguity: not a defect but a feature
-
Do standard NLP benchmarks hide LLM ambiguity failures?
When benchmark creators filter out ambiguous examples before testing, do they accidentally make it impossible to measure whether language models can actually handle ambiguity the way humans do?
this multiplicity is what benchmark design excludes
-
What three layers must discourse systems actually track?
Grosz and Sidner's 1986 framework proposes that discourse requires simultaneously tracking linguistic segments, speaker purposes, and salient objects. Understanding why all three are necessary helps explain where current AI systems structurally fail.
intentional structure is where social framing operates
-
Why do LLM persona prompts produce inconsistent outputs across runs?
Can language models reliably simulate different social perspectives through persona prompting, or does their run-to-run variance indicate they lack stable group-specific knowledge? This matters for whether LLMs can approximate human disagreement in annotation tasks.
the attempt to use LLMs to simulate multiple human perspectives fails because LLMs lack the stable social situatedness that makes interpretation group-specific
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments
- Can Large Language Models Capture Human Annotator Disagreements?
- We’re Afraid Language Models Aren’t Modeling Ambiguity
- Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
- Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?
- Language models show human-like content effects on reasoning tasks
Original note title
sentence interpretations are irreducibly multiple because social position and moral framing generate competing readings