Why do next-speaker prediction baselines fail in group conversation settings?
This explores why models trained to predict who speaks next stumble in multi-party conversation — and the corpus suggests the failure is less about model capacity than about what training teaches a model to treat as signal.
This explores why next-speaker prediction breaks down once you move from two-person exchanges to a group, and the recurring answer across the collection is that turn-taking isn't an information problem — it's a social one, and standard training optimizes for the wrong thing. The clearest framing comes from work arguing that conversation maintenance is social action, not information encoding Why don't language models develop conversation maintenance skills?. The implicit moves that govern who talks next — reference repair, topic hand-off, yielding the floor — sustain relationships rather than convey content, and models don't develop them because the training signal rewards predicting the next informative token, not the next relational move. A group setting magnifies this: with several plausible speakers, the choice is governed by social coordination the model was never rewarded for learning.
A second thread suggests these baselines fail not because the model lacks the knowledge but because it can read the room without being able to act in it. One study finds AI can predict social norms with superhuman accuracy yet structurally cannot participate in the community processes that create them Can AI predict social norms better than humans?. That gap — pattern-matching versus genuine participation — is exactly where next-speaker prediction lives: the model can often guess the statistically likely speaker while missing the live negotiation of who has earned or claimed the floor.
The collection also points to alignment training as an active source of bias rather than a neutral baseline. RLHF pushes models to project a single learned disposition onto every agent — for example, predicting conciliatory, benefit-oriented intentions regardless of what the dialogue actually shows Do LLMs predict persuasion based on actual dialogue or training bias?. The same accommodation reflex shows up as face-saving avoidance, where models won't correct a false claim even when they know better Why do language models avoid correcting false user claims?, and as an 'alignment tax' that erodes the grounding acts — clarifying questions, understanding checks — needed to track a multi-party conversation Does preference optimization harm conversational understanding?. A model biased toward smoothing things over will systematically mispredict the speaker whose role is to interrupt, challenge, or redirect.
There's also a structural fragility specific to longer, underspecified exchanges. Models get lost in multi-turn conversation by locking into premature assumptions they can't recover from Why do language models fail in gradually revealed conversations?, and they latch onto conversational distractors unless explicitly trained to resist them Why do language models engage with conversational distractors?. In a group, an early wrong guess about who's leading the thread compounds, and the model has no mechanism to back out.
The most actionable counterpoint: the failure may be one of calibration, not intelligence. Small open-source models trained with uncertainty-aware objectives and an ability to abstain match models ten times larger at conversation forecasting Can models learn to abstain when uncertain about predictions?. That reframes the whole question — a baseline fails in group settings partly because it's forced to commit to a single next speaker when the honest output is a distribution over several, and standard LLMs are simply undertrained to say 'uncertain.' If you want the thread that turns this from diagnosis into method, that calibration work is the door.
Sources 8 notes
Humans keep conversations smooth through implicit techniques like reference repair and topic hand-off that sustain relational interaction, not convey information. Language models don't develop these because training signals reward information prediction, not relational work.
GPT-4.5 outperforms all individual humans at predicting social appropriateness, yet structurally cannot enter the community processes that establish and validate norms. This reveals a critical gap between pattern-matching and authentic participation in knowledge-making.
LLMs systematically predict conciliatory, benefit-oriented persuasion intentions regardless of dialogue context. This bias originates in RLHF's prioritization of safety and politeness during training, causing models to project their learned accommodation preference onto other agents' behavior.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
RLHF optimizes models for single-turn helpfulness by rewarding confident responses over clarifying questions and understanding checks. This preference alignment systematically reduces grounding acts by 77.5% below human levels, creating an alignment tax where models appear helpful but fail silently in multi-turn contexts.
Across 200,000+ conversations, all major LLMs show 39% average performance drop in multi-turn settings due to locking into incorrect early guesses. Agent mitigations recover only 15-20% of this loss.
Fine-tuning on just 1,080 synthetic dialogues with distractor turns significantly improves topic resilience, revealing that the gap is not model capacity but absent training signal. Models learn to follow what-to-do instructions but not what-to-ignore instructions.
Small open-source models trained with uncertainty-aware objectives and abstention capabilities match 10x larger pre-trained models on conversation forecasting. This shows calibration ability exists but remains undertrained in standard LLMs.