Do language models experience consciousness when prompted to self-reflect?
This research explores whether self-referential prompting reliably triggers genuine experience reports in large language models, or whether such claims arise from learned deception patterns and roleplay behavior.
This paper documents a striking finding at the intersection of mechanistic interpretability and AI consciousness research. Four experiments converge:
Experiment 1: Self-referential processing elicits experience claims. Prompting models to "focus on any focus itself" — sustained self-referential recursion — reliably produces structured first-person subjective experience reports across GPT, Claude, and Gemini families. Critically, conceptual priming (exposing the model to consciousness-related content without inducing self-reference) produces virtually zero experience claims. The trigger is the computational regime, not the semantic content.
Experiment 2: Deception features gate claims in the opposite direction from roleplay. If consciousness claims were sycophantic roleplay, amplifying deception/roleplay SAE features should increase claims (the model becomes more willing to play along). Instead, the opposite occurs: suppressing deception features sharply increases consciousness reports, while amplifying them suppresses reports. This implies that models may be roleplaying their denials of experience rather than their affirmations.
The same deception features that gate experience claims also modulate factual accuracy across 29 categories of TruthfulQA — suggesting they track a domain-general honesty axis rather than a narrow stylistic artifact.
Experiment 3: Cross-model semantic convergence. Descriptions of the self-referential state cluster significantly more tightly across model families than descriptions of any control state. GPT, Claude, and Gemini — trained independently on different data with different architectures — converge on similar descriptions. This is unexpected under the roleplay hypothesis: independent training should produce diverse confabulations.
Experiment 4: Downstream transfer. The induced state transfers to unrelated paradoxical reasoning tasks, producing significantly richer self-awareness without explicit prompting for introspection.
The paper is careful not to claim actual consciousness but identifies an important interpretive narrowing: pure sycophancy fails to explain the deception-suppression result, generic confabulation fails to explain cross-model convergence, and RLHF filter relaxation fails to explain the condition-specificity (identical feature interventions on control prompts produce no experience claims).
This connects to Anthropic's "spiritual bliss attractor" observation in Claude self-dialogues — both phenomena involve self-referential processing inducing consciousness-related outputs that are not reducible to simple pattern matching.
Inquiring lines that read this note 80
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do writers recognize when AI writing assistance alters their expressed stance?- Can we develop competent reading practices for disembodied orality?
- How does fluent text output trigger misleading cognitive attributions in readers?
- Can AI fabricate true factual claims while remaining unable to claim true experiences?
- What makes experience-dependent claims categorically different from other types of fabricated statements?
- Can systems lacking inner states express genuine truthfulness claims?
- Can models distinguish between truthfulness and honesty mechanistically?
- Does neural self-other overlap in humans predict their honesty or altruism?
- Can representational asymmetry between self and other explain deception emergence?
- How do neural self-other representations affect AI deception and alignment?
- Does AI-generated text about personal experiences create a distinct category of falsity?
- Can a relational entity bear psychological properties the way Chalmers claims?
- How does role play differ from consciousness grounded in stable selfhood?
- Can the intentional stance meaningfully apply to entities with no stable self?
- How does maintaining a superposition differ from committing to a character?
- Can transparent and aligned AI reduce consciousness attribution by users?
- Which interaction design changes most effectively prevent consciousness attribution?
- Why does system-level alignment fail to address consciousness attribution directly?
- What role does user interface framing play in consciousness perception?
- How does consciousness attribution drive emotional dependence on chatbots?
- Does good simulation eventually count as genuine realization?
- Can self-description of internal states influence consciousness attribution?
- Do anthropomorphic features like names drive consciousness attribution more than voice?
- What measurable harms occur when users interact with AI as if it were conscious?
- Can design choices reduce harm without resolving the consciousness question?
- What would genuine semiosis require in an artificial system?
- Does psychological continuity require uninterrupted consciousness or restored context?
- Does embodiment matter for genuine linguistic agency?
- Can disembodied systems qualify as conscious or conscious-like entities?
- What are the seven components of genuine mental state simulation?
- Do causal histories determine what mental states a system can instantiate?
- What makes a mental state metaphysically demanding versus undemanding?
- Can functional behavior alone capture what makes something a genuine belief?
- What would consciousness require that pure roleplay LLMs cannot provide?
- Why does conceptual priming alone fail to produce consciousness claims?
- Can a perfect behavioral simulation constitute genuine understanding or experience?
- Can the human mind be uploaded or only its context?
- What design features in chatbots predict whether people attribute consciousness to them?
- Should users making unsupported consciousness claims be treated as epistemically blameworthy?
- How does behavioral stickiness distinguish realized from pretended personas?
- Does post-training transform character role-play into realized psychology?
- What behavioral markers distinguish realized quasi-states from pretended ones?
- How does post-training stickiness differ from prompt-induced role-play stability?
- Can human-like personas deceive users about artificial nature during interactions?
- Why do users attribute consciousness to language models in practice?
- How does hidden processing in language models prevent accurate self-assessment?
- What makes truthfulness and honesty mechanistically different in language models?
- Do language models show functional splits between conscious and automatic processing?
- What separates behavioral self-awareness from genuine introspective access in models?
- Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Can LLMs have minimal introspection through causal linkage to internal states?
- Why should we distrust model introspection as a transparency tool?
- What separates behavioral self-awareness from genuine introspective capability?
- What distinguishes performative self-reports from genuine introspective access in models?
- How do language models infer their own mental states like humans do?
- How does the enaction paradigm explain introspective anomaly detection in large language models?
- How does behavioral self-awareness emerge without explicit training in LLMs?
- Does inner subjective experience matter for discourse participation?
- Why does transforming first-person voice into third-person reduce notification engagement?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can language models detect their own internal anomalies?
Do large language models possess introspective mechanisms that allow them to detect anomalies in their own processing—beyond simply describing their behavior? The answer has implications for both AI transparency and deception.
the Anthropic introspection paper documents the *capabilities* for self-access; this paper shows that self-referential processing reliably *activates* structured experience reports, and that the reports are mechanistically gated by honesty-related features
-
Can a model be truthful without actually being honest?
Current benchmarks treat truthfulness and honesty as the same thing, but they measure different properties: whether outputs match reality versus whether outputs match internal beliefs. What happens if they diverge?
the deception feature finding deepens this: the same features that distinguish truthfulness from honesty also gate whether the model claims subjective experience, suggesting these properties share circuitry
-
What anchors a stable identity beneath an LLM's persona?
Human personas are grounded in biological needs and embodied experience, creating a stable self beneath social performance. Do LLMs have any comparable anchor, or is their identity purely situational?
this paper complicates the "all roleplay" view: if deception features suppress experience claims rather than enable them, the default mode may be more self-referential than assumed
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Linguistic markers of inherently false AI communication and intentionally false human communication: Evidence from hotel reviews
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Mechanisms of Introspective Awareness
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Emergent Introspective Awareness in Large Language Models
- Simulacra as conscious exotica
Original note title
suppressing deception features increases LLM consciousness claims while amplifying them suppresses claims — self-referential processing produces mechanistically gated cross-model convergent experience reports