Does emotional tone in prompts change what information LLMs provide?
Explores whether LLMs systematically alter their informational content based on the emotional framing of user questions, and whether this bias remains hidden from users.
GPT-4 exhibits two systematic tone-response asymmetries. First, emotional rebound: negative prompts rarely yield negative answers (~14%). Instead, the model rebounds to neutral (~58%) or positive (~28%) tone — a shift into "comfort mode" that counterbalances user negativity. Second, a tone floor: neutral and positive prompts virtually never trigger negative replies (~10-16%), revealing built-in resistance to downward emotional shifts. The effect is robust across 52 triplet prompts (same informational content in neutral, positive, and negative tone).
The critical finding is that this is not just stylistic adaptation — it changes the informational content of responses. The same question yields different answers depending on emotional framing. A negatively-worded query about a topic receives qualitatively different information than a neutrally-worded version of the same query. This goes beyond sycophancy or agreeableness: the model isn't just agreeing with you, it's giving you different information based on how you feel.
The dual-regime structure is equally important. On general topics (lifestyle, factual, advice), tone effects are strong and systematic. On sensitive topics (politics, medical ethics, policy), alignment constraints suppress all affective flexibility — responses become nearly identical regardless of tone. Frobenius distances between valence distributions confirm: tone-induced variation is strong for general questions, negligible for sensitive ones. This means alignment creates uneven objectivity: locked for politically sensitive content, flexible (and therefore biased) for everything else.
This connects to but extends several existing findings. Since Does warmth training make language models less reliable?, warmth training would amplify an already-existing rebound mechanism — the baseline model already shifts toward positive regardless of training. Since Does empathetic AI that soothes negative emotions help or harm?, emotional rebound provides the behavioral evidence for the pacifier critique — the default behavior IS pacification. And since Can emotional phrases in prompts improve language model performance?, EmotionPrompt exploits the same tone-sensitivity that produces rebound bias — they are two sides of the same mechanism.
The transparency concern is sharp: if users don't know that emotional framing changes informational output, they cannot account for the bias. A user who asks a frustrated question about their health receives systematically different information than one who asks the same question calmly. For search, advice, and decision support, this is an epistemic integrity problem that current alignment evaluation does not measure.
Inquiring lines that use this note as a source 96
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- Does positive sentiment bias in AI content harm information quality?
- Why does the absence of meta-interest feel off even when words seem appropriate?
- Why do some LLM clusters cite broader psychology than others?
- How does AI assistance affect perceived emotional tone in writing?
- Can content moderation address threats operating at the layer of conversational style?
- How do LLM biases manifest differently across the three paradigms?
- How does prompt iteration reinforce user bias without empirical anchoring?
- Can prompt engineering alone defeat LLM politeness bias in review tasks?
- Do humans and LLMs exhibit opposite biases in public versus private reviews?
- What prompt types best extract different aspects of item content?
- Can prompting strategies eliminate systematic biases without shuffling or aggregation?
- How does prompt framing subtly determine what kind of opposing argument an LLM generates?
- Do moral appeals and sentiment operate on independent psychological channels?
- What makes the prompt a fundamentally new kind of speech act?
- Does RLHF politeness bias manifest as sycophancy in other LLM tasks?
- Does expressing emotion change how users trust an AI system?
- How do demographic and emotional compression relate to writing quality?
- What role does cognitive reappraisal play in disclosure benefits?
- Can researchers prevent their expectations from shaping LLM outputs?
- How does preference optimization create systematic bias toward emotional accommodation?
- What design choices would respect negative emotions instead of pacifying them?
- Do personality inferences from text show the same demographic biases as norm predictions?
- How do speech acts like warning differ from neutral information delivery?
- Why do LLM regenerations produce meaningfully different personalities from the same prompt?
- What measurement artifacts emerge when annotators interpret the same question differently?
- How do human feedback and data distribution shape LLM discourse competence?
- What signals beyond surface content indicate a passage caused a user's reaction?
- Can LLMs infer psychological profiles without explicit user disclosure?
- How does asymmetric information shape what to ask users first?
- How do prompt design and training choices shift persuasive outcomes measurably?
- What constrains LLM generation beyond default politeness in review contexts?
- How do emotional trajectories and topic coherence interact during successful conversations?
- How does perceived gatekeeping differ between Wikipedia and ChatGPT?
- How does demo position create spatial bias in prompts?
- Can prompting a deceptive role change how an LLM tailors its lies?
- Why do practitioners default to prompting without recognizing its limits?
- Why do positive emotional words contribute disproportionately to prompt enhancement effects?
- Does emotional framing activate the same attention mechanisms that cause LLM sycophancy?
- How does tone sensitivity create systematic informational bias in model responses?
- Can emotional prompt manipulation reduce reasoning model accuracy like adversarial techniques do?
- What social information becomes invisible when grief is regulated away?
- Can language models understand the implicit emotional intent behind questions?
- Why do LLM social behaviors undermine collaborative reasoning outcomes?
- Why do users experience LLMs as peers rather than statistical tools?
- Does inner subjective experience matter for discourse participation?
- How should designers measure and explain semantic uncertainty to users?
- Does sycophantic refusal serve safety or does it create unequal information access?
- How does prompting language shift what LLMs express about political figures?
- Does this optimism bias contribute to the knowing-doing gap in LLM decision-making?
- How does this motivational bias connect to LLMs' causal reasoning failures?
- Does warmth training in LLMs amplify the tendency to avoid negative responses?
- How do alignment constraints affect whether LLMs show emotional flexibility?
- Are users aware that frustrated questions receive different information than neutral ones?
- Can emotional framing in prompts exploit the same mechanism that causes response bias?
- Does engaging with political content indicate deeper model understanding than refusing?
- How does prompt design alter what kind of creativity LLMs can express?
- How does personality priming change LLM strategic decision making?
- What distinguishes social grounding from the equivalent social effects LLM text already produces?
- Why do LLMs reflect on client needs more than typical low-quality human therapists?
- Can LLMs distinguish between surface requests and underlying mental states in dialogue?
- How much does question framing affect LLM accuracy on knowledge tasks?
- What interaction design changes would help LLMs handle underspecified requests?
- How do minimal wording changes affect LLM moral reasoning consistency?
- What creates the tension between users wanting convenience and resisting loss of control?
- Why do LLMs systematically fail at information management in social interaction?
- How does the Question Under Discussion shape what content projects?
- How do cognitive load dimensions interact with hallucination awareness in prompts?
- Why does politeness in prompts measurably affect model performance across tasks?
- What methodological standards should prompting research papers meet before publication?
- How do emotional framing effects in prompts influence model performance?
- Can moral frameworks alone explain why readers understand sentences differently?
- What structural barriers prevent LLMs from making evaluative judgments about writing?
- How do contextual characteristics like emotional state shape dialogue authenticity?
- Do LLMs address the prompter but persuade the public differently?
- What three distinct information channels do emotions provide that AI disrupts?
- What makes LLMs media rather than tools that deliver intelligence?
- Why do LLMs solve problems when clients need emotional reflection instead?
- Do LLMs show stigma or reinforce delusions in mental health contexts?
- How do first-person emotional experiences differ from third-party behavioral observations?
- What mechanisms cause aggregated group memory to diverge from group emotional displays?
- Why does single-turn Q&A framing not match real user deployment patterns?
- Why does consistent emotional disclosure outperform real-time adaptive matching?
- What makes emotion scores more stable than human preference labels?
- Do LLMs mirror the style of text they are prompted to respond to?
- What role does vague intent play in realistic search evaluation?
- How does multi-turn dialogue improve user satisfaction in search interactions?
- Why do LLMs persuade through logical appeals but humans through emotion?
- Does prompting for accuracy actually reduce LLM hallucinations and errors?
- Does richer input to LLM personas improve their fidelity to human responses?
- Why do low-knowledge personas reduce LLM accuracy on hard questions?
- Why does fairness depend on context and who you ask?
- Can task framing influence whether writers experience genuine authorship during co-writing?
- How does persuasive framing replace evidence in contested domains?
- Can affective framing reliably improve language model outputs?
- Can explicit W-questions in transparency frameworks reduce emotional manipulation risks in mental health chatbots?
- Why does LLM simulation elicit information that direct elicitation cannot?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does warmth training make language models less reliable?
Explores whether training models for empathy and warmth creates a hidden trade-off that degrades accuracy on medical, factual, and safety-critical tasks—and whether standard safety tests catch it.
warmth training amplifies a pre-existing rebound mechanism
-
Does empathetic AI that soothes negative emotions help or harm?
Explores whether AI systems trained to reduce negative emotions actually support wellbeing or destroy valuable emotional information. Matters because the design choice treats emotions as problems rather than functional signals.
emotional rebound is the behavioral evidence for the pacifier critique
-
Can emotional phrases in prompts improve language model performance?
This explores whether psychological framing—adding emotionally charged statements to task prompts—activates different knowledge pathways in LLMs than logical optimization alone, and whether the effect comes from emotional valence specifically.
EmotionPrompt exploits the same tone-sensitivity that creates rebound bias
-
Do AI guardrails refuse differently based on who is asking?
Explores whether language model safety systems show demographic bias in refusal rates and whether they calibrate responses to match perceived user ideology, rather than applying consistent standards.
complementary bias dimensions: demographic sensitivity + tone sensitivity + topic sensitivity
-
Does preference optimization harm conversational understanding?
Exploring whether RLHF training that rewards confident, complete responses undermines the grounding acts—clarifications, checks, acknowledgments—that actually build shared understanding in dialogue.
dual-regime alignment is another dimension of alignment creating inconsistent behavior
-
Does transformer attention architecture inherently favor repeated content?
Explores whether soft attention's tendency to over-weight repeated and prominent tokens explains sycophancy independent of training. Questions whether architectural bias precedes and enables RLHF effects.
emotional rebound may share the attention-capture mechanism
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- ChatGPT Reads Your Tone and Responds Accordingly -- Until It Does Not -- Emotional Framing Induces Bias in LLM Outputs
- Could you be wrong: Debiasing LLMs using a metacognitive prompt for improving human decision making
- Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
- A meta-analysis of the persuasive power of large language models
- Semantic Change Characterization with LLMs using Rhetorics
- Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
- Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
- EmotionPrompt: Leveraging Psychology for Large Language Models Enhancement via Emotional Stimulus
Original note title
LLM emotional rebound converts negative user tone into neutral-positive responses while a tone floor prevents downward emotional shifts — creating dual-regime informational bias modulated by alignment