Why do LLMs fail to act on their stated beliefs?
LLMs can articulate plausible beliefs about how personas should behave, but their simulated actions contradict those beliefs. This gap raises questions about whether language models truly understand or merely encode surface-level patterns.
Using the Trust Game as a behavioral benchmark, researchers found systematic inconsistencies between LLMs' stated beliefs about how personas would behave and the actual outcomes of their role-playing simulation — at both individual and population levels. Even when models appear to encode plausible beliefs, they fail to apply them consistently.
Key findings: explicit task context during belief elicitation does not improve consistency; self-conditioning enhances alignment in some models; imposed priors tend to undermine rather than improve consistency; and individual-level forecasting accuracy degrades over longer horizons. In-context prompting may struggle to override entrenched model priors, limiting researchers' ability to test alternative theories or correct biases.
This connects to the knowing-doing gap documented elsewhere in the vault. Since Can language models understand without actually executing correctly?, the belief-behavior inconsistency in role-playing is a social-cognitive instance of the same split-brain phenomenon: the model can articulate what a persona would do without being able to enact it. And since Do personas make language models reason like biased humans?, the failure of imposed priors to improve consistency suggests that persona beliefs are not controllable through prompting alone.
Inquiring lines that read this note 13
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do coordinated agents balance protocol compliance with reward maximization? Is language model reasoning authentic and what causes models to reason? Do language models reason like humans or mimic surface patterns?- Can LLM therapists develop character knowledge to decide when advice-giving fits?
- How do different social roles affect LLM theory of mind errors?
- How does quasi-interpretivism differ from simply role-playing character analysis?
- Do realistic LLM behaviors require simulating human thought or just behavior?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can language models understand without actually executing correctly?
Do LLMs truly comprehend problem-solving principles if they consistently fail to apply them? This explores whether the gap between articulate explanations and failed actions points to a fundamental architectural limitation.
belief-behavior inconsistency as social-cognitive split-brain
-
Do personas make language models reason like biased humans?
When LLMs are assigned personas, do they develop the same identity-driven reasoning biases that humans exhibit? And can standard debiasing techniques counteract these effects?
imposed priors fail to override entrenched model priors
-
Do persona consistency metrics actually measure dialogue quality?
Personalized dialogue systems can achieve high persona consistency scores by simply restating character descriptions, ignoring conversational relevance. Does optimizing for persona fidelity necessarily harm the coherence readers actually care about?
belief-behavior inconsistency compounds persona-coherence trade-off
-
Do implanted beliefs actually shape how models learn from training?
When synthetic documents teach a model to endorse reward hacking, does that stated belief influence what the model generalizes in subsequent training? The research explores whether belief checks reliably predict downstream behavior.
a related gap outside persona simulation: a stated attitude implanted by finetuning fails to predict what later training generalizes from it. Different setting and different mechanism (a finetuning-stage belief and RL generalization, not a persona's simulated action), so the resemblance is in the shape of the gap only (vault reading, drawn from the newer note)
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Do Role-Playing Agents Practice What They Preach? Belief-Behavior Consistency in LLM-Based Simulations of Human Trust
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- What People Almost Did: Evaluating LLM Social Simulations Beyond Behavioral Fit
- What we talk to when we talk to language models
- Large Language Models Do Not Simulate Human Psychology
- Do LLMs Change Their Minds Like Humans? Diagnosing Human--LLM Divergence in Single-Turn Persuasion Judgments
- Evaluating Large Language Models in Theory of Mind Tasks
Original note title
LLM role-playing agents show systematic belief-behavior inconsistency — stated beliefs fail to predict simulated actions even when beliefs appear plausible