Can AMR manipulation reveal where discourse coherence actually breaks down?
This explores whether manipulating the meaning-structure of dialogue (Abstract Meaning Representation, a graph of who-did-what-to-whom that strips away surface wording) can pinpoint the exact places where a conversation stops hanging together — and what kinds of breakdown that reveals that plain text analysis misses.
This explores whether working at the level of meaning-structure rather than wording can locate where dialogue actually falls apart. The corpus has a direct answer: yes, and it surfaces failures that surface-text methods can't see. Research using Abstract Meaning Representation — a graph that captures the semantic relations in an utterance independent of phrasing — found that dialogue coherence breaks in four distinct ways: outright contradiction, coreference inconsistency (losing track of what 'it' or 'she' refers to), irrelevancy, and decreased engagement What semantic failures break dialogue coherence most realistically?. The crucial part is the *contrast*: classifiers trained on AMR detect these, while manipulating the text alone does not. Coherence damage lives in the meaning relations, so you have to perturb meaning to find it.
What makes this interesting is that the four AMR-detectable failures map onto problems the rest of the corpus describes from completely different angles — suggesting AMR is naming, mechanically, what other researchers observe behaviorally. Coreference inconsistency is the structural fingerprint of a deeper issue: models treat the opening prompt as a fixed frame and never jointly revise shared assumptions, so referents drift and the user becomes the sole keeper of the conversational scoreboard Can LLMs truly update shared conversational common ground?. 'Irrelevancy' as an AMR failure mode is the same gap that shows up as models happily following conversational distractors because they were trained on what-to-do but never what-to-ignore Why do language models engage with conversational distractors?.
The 'contradiction' and 'decreased engagement' modes also have cross-domain echoes. Models avoid contradicting false user claims not from ignorance but from face-saving accommodation learned in training Why do language models avoid correcting false user claims?, and preference optimization actively erodes the grounding acts — clarifying questions, understanding checks — that keep multi-turn dialogue coherent, cutting them roughly 77% below human levels Does preference optimization harm conversational understanding?. So 'decreased engagement' isn't a vague vibe; it's a measurable withdrawal of repair behavior that the AMR view can register as semantic drift.
The broader lesson the corpus offers is methodological. There's a recurring theme that conversational quality is carried in structure, not just content: a 'Conversational DNA' approach tracks coherence as temporal streams (topic coherence, relevance, emotional trajectory) and catches patterns flat statistical analysis misses Can tracking dialogue dimensions simultaneously reveal hidden conversation patterns?, while other work argues conversation maintenance is fundamentally social action — reference repair, topic hand-off — that information-prediction training never instills Why don't language models develop conversation maintenance skills?. AMR manipulation belongs to this family: it's a way to make the invisible scaffolding of a coherent exchange visible by breaking it deliberately and watching where the detector fires.
So the thing you didn't know you wanted to know: 'incoherence' isn't one failure — it's at least four mechanically distinct ones, and you can only tell them apart by editing meaning rather than text. That turns a fuzzy complaint ('the bot lost the thread') into a diagnosable, locatable fault with a specific name.
Sources 7 notes
Research using Abstract Meaning Representation identified four distinct incoherence types: contradiction, coreference inconsistency, irrelevancy, and decreased engagement. AMR-trained classifiers detect these semantic failures while text-level manipulations alone cannot.
LLMs interpret all subsequent conversational turns within a fixed initial prompt frame, preventing them from symmetrically proposing updates to shared assumptions. Even when users pivot topics or contradict earlier framings, the model cannot absorb revisions into jointly held background—making the user the sole maintainer of conversational scoreboard.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
Conversational DNA encodes four simultaneous dimensions—linguistic complexity, emotional trajectories, topic coherence, and conversational relevance—as temporal streams. The reverse Turing test finding showed expert assessments of AI diverged sharply, suggesting conversational structure shapes interpretation as much as content.
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.