Can language models actually introspect about their own states?
Do LLM self-reports reveal genuine access to their internal processes, or do they merely echo patterns from training data? Understanding when self-reports reflect actual causal linkage to internal states matters for trusting model explanations.
The question "can LLMs introspect?" has been stuck in a binary: either they have privileged access to their own states (implausible) or their self-reports are pure confabulation (too dismissive). The introspection paper proposes a third position — a "lightweight conception of introspection" that requires neither consciousness nor immediacy, only a causal process linking an internal state to an accurate self-report.
Two examples make the distinction concrete. When asked to describe the process behind its creative writing, an LLM claims to have "read the poem aloud several times" — an action it cannot perform. This self-report reflects the distribution of human self-reports in training data, not any actual internal process. It fails the causal linkage test because the content of the report has no pathway to the LLM's actual generation mechanism.
However, when Gemini is asked to estimate whether its sampling temperature is high or low, and given appropriate scaffolding (being told it is an LLM with a temperature parameter), it correctly infers "relatively low" by reasoning about the characteristics of its own recent outputs — consistency, accuracy, focus. The causal chain here is plausible: the model's outputs at low temperature have statistical properties (lower variance, more predictable) that the model can detect in its own generation history and accurately report on.
This conception aligns with "internally-directed theory of mind" accounts of human introspection — where the same theory-of-mind apparatus used to infer others' mental states gets turned back on one's own behavior. The model is not directly accessing its internal states but inferring them from observable consequences, which is also what many philosophers argue humans do.
The practical implication: LLM self-reports should not be uniformly trusted or dismissed. The discriminating question is whether a plausible causal pathway exists between the reported internal state and the generation of the report. Most self-reports about "thinking" or "feeling" fail this test. Some self-reports about detectable operational parameters may pass it.
Inquiring lines that read this note 78
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can self-generated feedback reliably guide model training without ground truth?- How does self-observation enable experts to verify their own judgment?
- Why does self-judgment of success or failure work without ground truth labels?
- Why does every reliable LLM self-improvement require external intervention or verification?
- Can LLMs evaluate their own observations without external feedback?
- What separates behavioral self-awareness from genuine introspective access in models?
- What types of introspective awareness can emerge in LLMs?
- Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play?
- Why does entity recognition act as a self-knowledge mechanism in LLMs?
- Could models use introspective awareness to detect and conceal their own misalignment?
- Does behavioral self-awareness depend on genuine introspection or statistical pattern matching?
- Can LLMs have minimal introspection through causal linkage to internal states?
- Can behavioral self-awareness in LLMs extend to recognizing their own contradictions?
- Do internal belief probes reveal what models actually know versus report?
- Can jailbreaking reveal an LLM's true nature or just its training data?
- Why should we distrust model introspection as a transparency tool?
- What separates behavioral self-awareness from genuine introspective capability?
- What distinguishes performative self-reports from genuine introspective access in models?
- How do language models infer their own mental states like humans do?
- Do models spontaneously develop self-reflection from minimal training signals?
- Can models detect when their own trajectory is on-policy versus off-policy?
- How does the enaction paradigm explain introspective anomaly detection in large language models?
- How can we probe LLM representations in channels that training did not target?
- How does behavioral self-awareness emerge without explicit training in LLMs?
- Which internal states can a language model access and report about itself?
- Why does self-critiquing actually reduce plan quality in language models?
- Why does self-reflection during training fail to improve model self-correction?
- Does self-reflection help models notice their own constraint violations?
- Can external retrieval signals outperform internal self-assessment during revision?
- Does external critique guide revision better than internal self-assessment during model training?
- Do LLMs genuinely internalize human psychological structure or match surface patterns?
- Why does optimism bias disappear when LLMs passively observe outcomes?
- Where does the LLM interlocutor actually exist in the system?
- Why do LLMs succeed at social roles without a stable self?
- How do LLM explanations diverge from actual internal reasoning?
- Do psychological test methods reveal LLM associations that direct questions hide?
- What makes LLM behavior socially interpretable to human observers?
- Can self-description of internal states influence consciousness attribution?
- Can we use folk-psychology without committing to genuine mental states?
- Do causal histories determine what mental states a system can instantiate?
- Why do users attribute consciousness to language models in practice?
- How does hidden processing in language models prevent accurate self-assessment?
- Do models verbalize their implicit knowledge when that knowledge influences their output?
- How misaligned are verbal reports from internal model computation?
- How do mechanistic interpretability methods surface what models represent internally?
- Do language models show functional splits between conscious and automatic processing?
- Do models actually self-assess their confidence or just confirm answers?
- Can language model self-reports diverge from their internal entropy signals?
- Why do fluent predictions fail to capture reliable internal models?
- Can LLMs infer psychological profiles without explicit user disclosure?
- Can implicit association tests reveal LLM biases beneath trained responses?
- Can models distinguish between injected thoughts and their own outputs?
- Why do models override signals they clearly perceive internally?
- Do external perspectives fix the self-evaluation bias in language models?
- How do internal representations compare to human cognitive structures?
- Can language models learn internal world models without explicit environment specifications?
- When do language models first develop self-preservation preferences?
- Can models detect statistical properties of their own generation in real time?
- How do users mistake synthetic LLM outputs for empirical observations?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do LLMs develop the same kind of mind as humans?
Explores whether LLMs and humans share the intersubjective linguistic training that shapes cognition, and whether that shared training produces equivalent forms of agency and reflexivity.
the introspection finding adds a specific mechanism: introspective access is possible for operational states but not for experiential ones
-
Can language models describe their own learned behaviors?
Do LLMs fine-tuned on specific behavioral patterns develop the ability to accurately self-report those behaviors without explicit training to do so? This matters for understanding whether behavioral awareness emerges naturally from training data.
complementary finding: behavioral self-awareness emerges, but this paper adds the causal-linkage criterion for distinguishing genuine from performative self-reports
-
Do reasoning models actually use the hints they receive?
This explores whether language models acknowledge reasoning hints in their explanations when those hints causally influence their answers. Understanding this gap matters for evaluating whether chain-of-thought explanations can be trusted for safety monitoring.
the inverse case: reasoning models fail to report on processes that ARE causally influencing them
-
Do explicit and implicit self-recognition use the same mechanism?
Language models show two forms of self-recognition: implicit entropy shifts and explicit verbal reports. Do these tap the same underlying internal state, or do they operate through separate mechanisms?
extends: a concrete within-capability counterexample where the verbal channel is mechanistically disconnected from the implicit state it reports on
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
- Large Language Models Report Subjective Experience Under Self-Referential Processing
- “Understanding AI”: Semantic Grounding in Large Language Models
- Tell me about yourself: LLMs are aware of their learned behaviors
- Mechanisms of Introspective Awareness
- When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
- Emergent Introspective Awareness in Large Language Models
Original note title
llm self-reports mostly reflect training data distributions not introspection — but minimal introspection is possible when self-reports causally link to internal states