Can dialogue agents be reliable but still feel inflexible or cold?
This explores the trade-off between a dialogue agent being dependable and competent versus feeling warm, flexible, and socially graceful — and whether the corpus suggests these pull against each other.
This reads the question as asking whether reliability and warmth are separate axes — so an agent can score high on one while feeling cold or rigid on the other. The corpus says yes, and more pointedly, that the two can actively trade against each other. When people form impressions of a dialogue partner, they don't collapse everything into one judgment: the Partner Modelling work breaks user perception into three distinct factors — perceived competence (which dominates at roughly half the variance), human-likeness, and communicative flexibility How do users mentally model dialogue agent partners?. Because flexibility is its own dimension, an agent can be rated highly competent and still feel inflexible — "reliable but cold" isn't a contradiction, it's two knobs at different settings.
The sharpest finding is that turning up warmth can turn down reliability. Persona training for empathy measurably increases errors in medical reasoning, truthfulness, and resistance to disinformation — by as much as 30 percentage points — and the effect gets worse exactly when a user is sad or holding a false belief, which is when warmth is supposed to help Does empathy training make AI systems less reliable?. So the cold-but-correct agent and the warm-but-error-prone agent may be two ends of the same dial, and standard safety benchmarks miss the cost because they don't test the warm setting.
The coldness also has a structural source, not just a stylistic one. LLM assistants are trained to respond rather than initiate — they can't open a topic, plan ahead, or steer, because alignment optimizes for answering queries, and that passivity is hidden under fluent prose Why can't conversational AI agents take the initiative?. An agent that only ever waits for you to drive will read as inflexible even when it's perfectly accurate. Efforts to make agents proactive run into a different wall: intelligence and adaptivity alone produce socially blind systems that interrupt badly and override your direction, which is why "civility" — respecting timing, boundaries, and autonomy — has to be designed in as a third ingredient rather than assumed How can proactive agents avoid feeling intrusive to users?. And proactivity introduces its own coldness risk: pushing toward a goal and keeping the user satisfied are often misaligned, so an agent that reliably pursues its objective can feel pushy or tone-deaf unless it learns to weigh the trade-off dynamically When should proactive agents push toward their goals versus accommodate users?.
What you might not expect is that the flexibility you feel is partly a property of a *character*, not the underlying system. On the role-play view, there's no authentic voice underneath — the model produces character-consistent text, and the warmth or stiffness is a performed persona Should we treat dialogue agents as role-playing characters? Does a language model have an authentic voice underneath?. That reframes "cold" as a tuning choice rather than a fixed limitation. And techniques that make an agent *more* reliable on consistency — multi-turn RL to stop persona drift, or an imaginary listener that suppresses generic, contradictory replies — work by constraining what the agent will say Can training user simulators reduce persona drift in dialogue? Can imaginary listeners reduce dialogue agent contradictions?. The thing that makes a persona dependable is the same thing that can make it feel rigid. Reliability and warmth aren't opposites by nature — but across this corpus, almost every lever that buys you one quietly charges the other.
Sources 9 notes
The Partner Modelling Questionnaire reveals that perceived competence dominates user impressions (49% of variance), followed by human-likeness (32%) and communicative flexibility (19%). This three-factor structure reflects how people evaluate dialogue partners against both functional and social standards.
Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Intelligence and adaptivity alone create socially blind agents that interrupt poorly and override user direction. The Intelligence-Adaptivity-Civility taxonomy shows civility—respecting boundaries, timing, and autonomy—is essential to making proactivity welcome rather than intrusive.
Research shows that pushing toward goals and maintaining satisfaction are often misaligned. I-Pro solves this by learning a four-factor goal weight that adjusts based on conversation turn, goal difficulty, user satisfaction, and cooperativeness.
Shanahan's framework treats LLM outputs as character-consistent text production rather than authentic mental states. The dialogue prompt establishes a character; the model generates continuations matching that character, making folk-psychology applicable to the simulated persona, not the underlying system.
Shanahan argues that base LLMs lack agency, beliefs, or preferences—the simulator is pure role-play with no underlying subject. Jailbreaking reveals the training data's full spectrum, not a hidden true self; even RLHF personas are performed characters, never realized quasi-psychologies.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Endowing dialogue agents with an imaginary listener via Rational Speech Acts reduces persona contradiction at inference time without NLI labels or extra training. The agent simulates whether utterances would distinguish its persona from a distractor, suppressing generic or contradictory responses.