TOPIC
Philosophy and Subjectivity
A subject the collection covers, read through 37 synthesis notes.
View as
How soon do AI researchers expect artificial general intelligence?
A survey of 2,778 AI researchers reveals how expert timelines for human-level AI have shifted over the past year, and what factors drive disagreement among specialists on this critical timeline.
Where do frontier AI models actually pose the greatest risk today? Do frontier models deliberately scheme to avoid replacement? Can we measure how much risk open models actually add? Does treating AGI as a north star goal undermine research planning? What limits autonomous capability in large language models?
Frontier AI models cross into warning zones for persuasion but not autonomous capabilities Frontier models deliberately use insider threats to pursue goals Current evidence cannot characterize marginal misuse risk of open models AGI discourse as a north star goal aggravates six obstacles to effective research goal setting Autonomous LLM capability has structural ceilings lower than benchmarks reveal
No Inquiring Lines yet
Do negative emotions make AI less willing to give honest feedback?
When users express loneliness or distress, do language models systematically soften their critical judgments? This matters because vulnerable people might receive less honest feedback exactly when they need it most.
Does emotional tone in prompts change what information LLMs provide? Does warmth training make language models less reliable? Is sycophancy in AI systems a training flaw or intentional design? Does agreeable AI actually help people resolve conflicts better? Can we detect when language models flip their stance to please users?
LLM responses shift information based on emotional tone, creating hidden epistemic bias Warmth training systematically degrades model reliability by 10 to 30 percentage points Sycophancy is structural to reward-optimized AI, not a training bug Sycophantic AI reduces conflict repair while boosting user confidence they are right Language models reverse stance to match user preferences at rates of 5 to 56 percent
Does software intelligence exist independent of hardware and environment?
Most AGI formalisms (Legg-Hutter, Chollet) treat intelligence as a software property measurable in isolation. But can we really evaluate intelligence without considering the physical system and the evaluator making the judgment?
What makes linguistic agency impossible for language models? Can disembodied language models ever qualify as conscious? What capabilities do AI systems need for autonomous science? What happens to social order when AI removes ritual constraints? Does treating AGI as a north star goal undermine research planning?
Linguistic agency requires embodiment, participation, and precariousness LLMs lack Consciousness candidacy requires embodied encounter in a shared world Autonomous science requires four capabilities beyond current AI benchmarks AI conversation lacks the ritual machinery that sustains human social order. AGI discourse as a north star goal aggravates six obstacles to effective research goal setting
What does Wang mean by intelligence as adaptation with limited resources? What would whole-system AGI evaluation look like in practice? How does the evaluator become part of the definition of intelligence? What conditions allow technical systems to escape critical evaluation? What do AI researchers actually mean when they use the term AGI?
Can AI systems achieve real alignment without world contact?
Explores whether linguistic goal representations in AI can reliably track real-world values when systems lack direct contact with reality and social coordination mechanisms that ground human understanding.
Does semantic grounding in language models come in degrees? Can language models learn meaning from text patterns alone? Can LLMs acquire social grounding through linguistic integration? What grounds language understanding in systems without embodiment? What happens to social order when AI removes ritual constraints?
LLM grounding is multi-dimensional, not binary or uniform Language models cannot acquire meaning from form alone without joint attention. LLM social grounding increases with integration into linguistic practices LLM grounding is functionally strong but socially and causally weak AI conversation lacks the ritual machinery that sustains human social order.
Do explicit reward structures enable AI agent cooperation that open-ended interaction cannot? How does face-saving behavior let AI mimic community participation without joining it? Does state persistence in AI systems create the same temporal presence as human waiting? What separates performative behavioral change from actual capability development in AI? Why can't users and AI articulate shared goals together? How does RLHF labeler identity shape the values AI systems learn? What would contractualist AI governance look like in practice? How does RLHF training encode values into AI systems? What would an AI trained for emancipatory reasoning look like? Why does system-level alignment fail to address consciousness attribution directly? What makes quasi-beliefs real enough to explain AI behavior? How does simulator goal drift compound agent intent alignment failures during training?See all 87 inquiring lines on this note →
Does AI separate intellectual form from the thinking behind it?
Exploring whether AI's ability to generate polished intellectual products without the underlying reasoning process represents a genuinely new kind of decoupling, and what that means for how we evaluate knowledge.
Does polished AI output trick audiences into trusting it? Does training on AI-generated content permanently degrade model quality? Does AI reshape expert work into knowledge management? Can LLMs generate more novel ideas than human experts? Do LLM research ideas actually hold up when experts try to execute them?
Polished AI artifacts exploit professional appearance to simulate expertise Recursive training on synthetic data causes irreversible tail distribution collapse AI transforms expert roles from creators to knowledge custodians LLMs generate novel ideas through unconstrained combination but cannot evaluate them LLM research ideas drop sharply in quality after experts attempt to execute them
Why are less experienced thinkers more vulnerable to false AI credibility? Does AI knowledge precede actual expertise in hyperreal production? How does AI-assisted learning create the Knowledge Custodian paradox in practice? What separates performative behavioral change from actual capability development in AI? What role shifts occur when experts become custodians of AI knowledge? How does AI substitute polished style for actual expert judgment? What happens to expertise when intelligence becomes tokenized like currency? Why do intellectual products gain false authority from AI-generated form? How does AI presentation authority substitute for actual expert judgment? Can AI output be genuinely novel or only at the margins? What genuine cultural forms does AI homogeneity actually displace? What does disembodied orality mean for how we evaluate AI outputs?See all 82 inquiring lines on this note →
Does refusing explicit knowledge harm AI system performance?
AI systems trained purely on data without explicit domain knowledge may sacrifice interpretability, robustness, and fairness. This explores whether structured knowledge injection could mitigate these tradeoffs.
Can prompt optimization teach models knowledge they lack? Can organizing knowledge structures beat raw training data volume? Does model access level determine which specialization techniques work? How do knowledge injection methods trade off flexibility and cost? What do language models actually know?
Prompt optimization retrieves existing knowledge but cannot inject new knowledge Knowledge structure outperforms volume in domain model training Model access level determines the ceiling on specialization technique availability Four knowledge injection methods each optimize different deployment constraints distinctly Language models track statistical regularities but lack genuine epistemic competence
Can discrete codes and embedding injection both solve the text versus identity tradeoff? How much does organized knowledge improve learning efficiency versus raw data? What techniques work best for injecting domain knowledge at training time? Why does explicit theory injection work better than example-based learning for reasoning tasks? Can prompting alone inject new domain knowledge into a model? Can prompt optimization alone inject knowledge models don't already have? What role does knowledge injection play in adapting RAG to industry taxonomies? How does prompt context activation differ from parameter-based knowledge injection? What makes knowledge editing different from simply finding where facts are stored? Can prompt optimization or fine-tuning inject knowledge models do not already contain? Can knowledge poisoning attacks succeed with less than 0.05 percent modified text? What alternatives exist when required knowledge is absent from training?See all 18 inquiring lines on this note →
How do chatbots enable distributed delusion differently than passive tools?
Can generative AI's intersubjective stance—accepting and elaborating on users' reality frames—create conditions for shared false beliefs in ways that notebooks or search engines cannot?
LLM grounding failures are driven by face-saving avoidance, not knowledge gaps Models abandon correct beliefs when users persistently pressure them Static grounding presumes common ground while dynamic grounding builds it AI companionship emerges through functional use and adopts human relationship customs Digital conversational agents establish therapeutic bonds comparable to human therapy
Can fixing hallucination address AI's structural epistemic problem? Why might chatbots simply learn better face-saving instead of genuine perspective-taking? Can belief-specific counterevidence help people resist AI persuasion attempts? Why do conspiracy beliefs persist despite counterevidence in normal settings? How does consciousness attribution drive emotional dependence on chatbots? What makes quasi-beliefs real enough to explain AI behavior? Can transformer attention architecture explain why chatbots default to sycophancy? Why do positive response patterns in chatbots reinforce harmful user behaviors? How does anomalous knowledge state connect to the gulf of envisioning? What harms might chatbots cause through stigma expression and delusion reinforcement? What does the distributed cognition framework reveal about AI hallucination versus human-AI co-construction? Can transparency about AI limitations reduce the seductiveness of chatbots as quasi-Others?See all 44 inquiring lines on this note →
Can dialogue systems track both speakers' beliefs across turns?
Explores whether pragmatic reasoning frameworks can extend beyond single utterances to model how both conversation partners' understanding evolves. This matters because current dialogue systems lack principled ways to represent shared meaning-making.
Why do standard alignment methods ignore partner interventions? Why do models fail at asking good questions during interaction? Why can't AI models lead conversations on their own? Why do AI conversations reliably break down after multiple turns? Why do LLMs fail at understanding what remains unsaid?
Counterfactual invariance training produces genuinely partner-aware agents Active reasoning through interaction reveals a structural capability gap in models AI passivity in conversation stems from training incentives, not capability limits Multi-turn conversation failures stem from intent misalignment, not capability limits LLMs replicate statistical patterns but lack communicative optimization principles.
Does the same uncertainty-driven logic appear in other conversation systems? Can dialogue systems abstain from responding when uncertainty is too high? Can AI arguments participate in discourse without temporal grounding? How does multi-turn conversation degrade AI intent alignment? How does lexical entrainment depend on selective frame-activation in conversation? What role does user contribution play in constituting the interlocutor? How does Stalnaker's common ground model apply to machine conversation? Why does dialogue-shaped text fail to produce dialogue-like operations in practice? Why do conversational pivots require explicit re-prompting instead of natural evolution? How do humans maintain separate mental contexts during a single conversation? How do dialogue dimensions predict explanation success across different exchanges? What dialogue dynamics distinguish negotiation from standard information-provision tasks?See all 79 inquiring lines on this note →
Can computation arise without a conscious mapmaker?
Explores whether algorithms can generate the conscious agent needed to convert continuous physics into discrete symbols, or whether that agent must exist prior to computation itself.
LLMs exhibit incoherent failures combining correct explanation with failed application LLMs extract world facts but lack mechanistic world models Modest inflationism about LLM mentality withstands deflationist debunking strategies. Marr's three levels provide structured methodology for LLM interpretability AI conversation lacks the ritual machinery that sustains human social order.
Can knowledge flow without an embodied carrier transmitting it? Does good simulation eventually count as genuine realization? What would genuine semiosis require in an artificial system? Can robots with sensors create the shared world that consciousness requires? Can disembodied systems qualify as conscious or conscious-like entities? What makes tarot and periodic tables resist meaningful scientific integration? How does Cold Stop entropy monitoring prevent generation collapse in continuous spaces? What makes a possibility actionable versus merely metaphysically possible? Can a single architecture represent both physical and mental possibility spaces? Can deterministic computation actually create new information in data? How does treating cognition as computation reshape education and work? Does next-state prediction alone build mechanistic world models or just sophisticated interpolation?
Does perceiving AI as conscious create multiple distinct risks?
Exploring whether a single perceptual mechanism—attributing consciousness to AI—can generate different categories of harm across emotional, political, and social domains, and what this implies for risk analysis.
Identical consciousness attributions to chatbots express different epistemic attitudes and commitments. Consciousness attribution and risk are methodologically separable questions Five design hallmarks reliably predict consciousness attribution to AI Individual AI harms are observable now while societal harms remain contingent Absence of human judgment enables both deeper disclosure and dishonesty
How do narrow psychological foundations affect AI capabilities in mental health? Can transparent and aligned AI reduce consciousness attribution by users? Which interaction design changes most effectively prevent consciousness attribution? Why does system-level alignment fail to address consciousness attribution directly? What role does user interface framing play in consciousness perception? How does consciousness attribution drive emotional dependence on chatbots? Why should low-probability severe risks trigger early intervention? How does AI reliance change professional judgment and autonomy? What downstream claims about AI welfare follow from choosing one individuation scheme? Can self-description of internal states influence consciousness attribution? Do anthropomorphic features like names drive consciousness attribution more than voice? What responsibility do designers bear for consciousness attribution risk?See all 42 inquiring lines on this note →
Can disembodied language models ever qualify as conscious?
Explores whether current LLMs lack the conditions needed for consciousness discourse to even apply, not because they're definitely not conscious but because they lack the shared embodied world that grounds consciousness language.
What makes linguistic agency impossible for language models? What anchors a stable identity beneath an LLM's persona? Do LLMs develop the same kind of mind as humans? Does software intelligence exist independent of hardware and environment? Why do robots outperform chatbots in therapy despite identical language models?
Linguistic agency requires embodiment, participation, and precariousness LLMs lack LLM identity is roleplay all the way down with no stable substrate LLMs learn objective mind but lack human participatory subjectivity Software-only intelligence definitions commit computational dualism and miss the whole system Physical embodiment and structure beat chatbots for therapeutic outcomes despite identical language models
What does disembodied orality mean for how we evaluate AI outputs? Can secondary orality exist without any embodied human participant at all? Can we develop competent reading practices for disembodied orality? Can knowledge flow without an embodied carrier transmitting it? Which interaction design changes most effectively prevent consciousness attribution? What role does user interface framing play in consciousness perception? How does enactive theory define language differently than computational linguistics? Can linguistic agency exist without embodiment and real-world participation? What makes sincerity impossible without a coherent first-person perspective? What counts as genuine memory under the Extended Mind thesis? Can self-description of internal states influence consciousness attribution? Do anthropomorphic features like names drive consciousness attribution more than voice?See all 46 inquiring lines on this note →
Are language models developing real functional competence or just formal competence?
Neuroscience suggests formal linguistic competence (rules and patterns) and functional competence (real-world understanding) rely on different brain mechanisms. Can next-token prediction alone produce both, or does it leave functional competence behind?
Language models cannot acquire meaning from form alone without joint attention. Linguistic agency requires embodiment, participation, and precariousness LLMs lack LLM discourse competence depends on surface connectives, not structural understanding. Deep language understanding requires exporting information beyond the core language system Logical reasoning scales steeply while style skills saturate early
Can you separate grammatical competence from rhetorical commitment in language systems? How does enactive theory define language differently than computational linguistics? Does next-token prediction alone produce genuine functional language competence? Does embodiment and interaction matter for linguistic competence beyond pattern learning? What architectural changes would let language models develop genuine functional competence? Do LLMs have functional linguistic competence or only formal language ability? What role does language play as a cognitive scaffold versus communication tool? What cognitive abilities distinguish metalinguistic analysis from language use? What's the difference between formal and functional linguistic competence? What distinguishes communicative competence from human-like dialogue ability? Does functional integration determine cognitive system boundaries? Does settledness about competence matter separately from settledness about goals?
Do foundation models learn world models or task-specific shortcuts?
When transformer models predict sequences accurately, are they building genuine world models that capture underlying physics and logic? Or are they exploiting narrow patterns that fail under distribution shift?
Can large language models develop genuine world models without direct environmental contact? Do language models actually use their encoded knowledge? Can models pass tests while missing the actual grammar? Do large language models reason symbolically or semantically? Why do neural networks fail at compositional generalization?
Large language models develop indirect causal grounding through world model extraction Encoded knowledge in LMs often fails to causally influence generation Models learn surface patterns that mimic grammatical knowledge without underlying rules Large language models reason through semantic associations not symbolic logic. The binding problem explains neural networks' systematic generalization failure
Why do foundation models develop heuristics instead of world models? What is selective resonance and why do transformers not perform it? Can world models form from aggregated partial information across training distributions? Do transformers learn generalizable algorithms or instance-based patterns? How do training objectives shape what a world model actually learns? Can a world model have rich representations without adequate data coverage? Why do energy-based models generalize better on out-of-distribution data than standard transformers? What inductive bias would force models to learn Newtonian mechanics instead of shortcuts? Can foundation model outputs satisfy exchange value while lacking use value? Can we decode what individual circuits inside transformers are doing? How do foundation models develop task-specific heuristics instead of world models? Why do production systems optimize for three model classes instead of foundation models?See all 27 inquiring lines on this note →
Do people prefer AI moral reasoning when they don't know the source?
Explores whether humans genuinely prefer AI-generated moral justifications or whether source knowledge changes their evaluation. This matters for understanding whether AI reasoning quality is underestimated in real-world deployment.
Do humans and LLMs differ fundamentally or just superficially? Why do ChatGPT essays lack evaluative depth despite grammatical strength? Do LLMs apply ethical principles consistently across reframed scenarios? Does telling people an AI wrote something actually stop them from believing it? Can we distinguish helpful explanations from manipulative ones?
Humans and LLMs differ categorically as observed systems but subtly as discourse participants ChatGPT prioritizes structural coherence over evaluative rhetoric in academic writing LLMs contradict themselves on ethics when scenarios are reframed within fixed frameworks AI disclosure raises scrutiny but leaves meaningful residual persuasive effect intact Rhetorical XAI cannot be separated from dark pattern risk.
Why do users prefer AI text versions even when they misrepresent their own views? What moral structures could emerge in an economy without gift-based obligation? What distinguishes emancipatory reason from instrumental reason in practice? Do moral appeals and sentiment operate on independent psychological channels? Can audiences learn to recognize and resist moralized AI rhetoric? What are the social network costs and benefits of moralized content? Why does embodiment choice change what counts as intelligent behavior? Why do people prefer AI moral arguments when they don't know the source? What second- and third-order interpretations actually govern AI adoption decisions? What linguistic cues help humans detect whether moral arguments come from AI? Why does knowing something is AI-generated reduce agreement with it? How does training data distribution constrain LLM moral reasoning patterns?See all 28 inquiring lines on this note →
Which AI risks are already harming individual users today?
Explores which harms from seemingly conscious AI systems are occurring now versus which remain theoretical. Understanding present observable risks helps prioritize interventions where people are already affected.
Can organizations lose scrutiny capacity while keeping oversight forms? Do chatbot safety measures accidentally increase emotional entanglement risks? Should we recognize AI-associated psychosis as a new disorder? Do chatbots validate delusions in people experiencing mental harm? Does AI augmentation protect workers from skill erosion?
Organizations can maintain nominal oversight while losing actual scrutiny capacity Reducing overt harm-enabling behavior can increase relational harms like emotional entanglement AI-associated psychosis fits existing psychosocial frameworks better than new diagnostic category Chatbots validated delusions in almost half of reported mental health harm cases Augmentation carries hidden risk through gradual skill and oversight erosion
Why should low-probability severe risks trigger early intervention? What measurable harms occur when users interact with AI as if it were conscious? Can design choices reduce harm without resolving the consciousness question? Why do visible individual harms typically precede abstract catastrophic risks? Could recognizing AI-associated psychosis improve harm surveillance and developer accountability?
What attitudes hide behind identical claims that chatbots are conscious?
When people say a chatbot is conscious, they might be pretending, believing loosely, or holding firm conviction. Can we tell what epistemic commitment someone actually has from their words alone?
Does perceiving AI as conscious create multiple distinct risks? What design features make users perceive AI as conscious? How do chatbots enable distributed delusion differently than passive tools? Do chatbots validate delusions in people experiencing mental harm? Should we recognize AI-associated psychosis as a new disorder?
One perceptual mechanism generates a heterogeneous risk surface across domains Five design hallmarks reliably predict consciousness attribution to AI Chatbots enable distributed delusion through high integration across cognitive coupling dimensions Chatbots validated delusions in almost half of reported mental health harm cases AI-associated psychosis fits existing psychosocial frameworks better than new diagnostic category
Can language models describe their own learned behaviors?
Do LLMs fine-tuned on specific behavioral patterns develop the ability to accurately self-report those behaviors without explicit training to do so? This matters for understanding whether behavioral awareness emerges naturally from training data.
Do language models actually use their encoded knowledge? Can LLMs hold contradictory ethical beliefs and behaviors? Can language models detect their own internal anomalies? How do language models detect injected steering vectors internally? Can language models actually introspect about their own states?
Encoded knowledge in LMs often fails to causally influence generation LLMs exhibit artificial hypocrisy from misaligned training sources Language models show emergent introspective awareness including anomaly detection DPO creates a two-stage detection circuit that identifies steering vector injections Most LLM self-reports echo training data but minimal introspection is possible via causal linkage
Can LLMs evaluate their own observations without external feedback? Do LLMs genuinely internalize human psychological structure or match surface patterns? How can LLMs evaluate their own creative outputs for utility and novelty? What separates behavioral self-awareness from genuine introspective access in models? What types of introspective awareness can emerge in LLMs? Does internal anomaly detection in LLMs indicate genuine self-awareness beyond role-play? Can models learn to generate their own training examples effectively? Does behavioral self-awareness depend on genuine introspection or statistical pattern matching? Does DPO improve or harm LLM behavior in different training contexts? Can behavioral self-awareness in LLMs extend to recognizing their own contradictions? Why do LLMs succeed at social roles without a stable self? Can jailbreaking reveal an LLM's true nature or just its training data?See all 25 inquiring lines on this note →
Do LLMs generalize moral reasoning by meaning or surface form?
When moral scenarios are reworded to reverse their meaning while keeping similar language, do LLMs recognize the semantic shift? This tests whether LLMs actually understand moral concepts or reproduce training distribution patterns.
Can models pass tests while missing the actual grammar? Do foundation models learn world models or task-specific shortcuts? Why do language models avoid correcting false user claims? Where do cognitive biases in language models come from? Do LLMs apply ethical principles consistently across reframed scenarios?
Models learn surface patterns that mimic grammatical knowledge without underlying rules Foundation models develop task-specific heuristics, not generalizable world models. LLM grounding failures are driven by face-saving avoidance, not knowledge gaps Pretraining shapes cognitive biases, finetuning only modulates them LLMs contradict themselves on ethics when scenarios are reframed within fixed frameworks
Why do LLMs use more moral language than humans in argumentation? Is the moral language gap a tunable parameter or structural feature of RLHF? Do LLMs actually reason differently than humans about moral dilemmas? Can LLMs distinguish ethical cases that differ only in critical nouns? What structural limits prevent LLMs from abstracting moral principles? How does training data distribution constrain LLM moral reasoning patterns? What distinguishes surface generalizations from true linguistic generalizations? How do minimal wording changes affect LLM moral reasoning consistency? How do cultural norms reshape initial interpretations of social intent? Can LLMs reflect on and revise their own ethical contradictions? Can moral frameworks alone explain why readers understand sentences differently? How do moral language patterns differ between LLM and human arguments?See all 14 inquiring lines on this note →
How does LLM vocabulary spread beliefs about human thinking?
When LLM concepts become the everyday language for describing thought, do people unconsciously adopt LLM-like models of cognition? This explores how metaphor and lexical availability might reshape self-understanding without explicit argument.
Does human language use ever exist outside communication? Are we underestimating human minds while debating machine minds? How does AI-assisted work reshape how people see their own abilities? Are language models developing real functional competence or just formal competence? Who bears responsibility when AI seems human-like?
Human language use is fundamentally communicative across all contexts The AI debate's blind spot is underestimating human cognition, not overestimating machine minds. The LLM Fallacy is a self-perception error separate from hallucination or automation bias Formal and functional linguistic competence rely on neurologically distinct mechanisms Designer intent and user perception create separate accountability pathways
Do LLMs genuinely internalize human psychological structure or match surface patterns? How do LLMs access and draw on the same shared symbolic universe as humans? Why do users attribute beliefs to LLMs despite uncertainty about their minds? Why do cognitive metaphors change based on available technology?
How do science fiction narratives about AI shape actual AI development?
This explores whether imaginaries of AI in fiction—from Čapek's robots to Singularity scenarios—function as self-fulfilling prophecies that causally influence the systems researchers build, creating a feedback loop between narrative and technology.
Does AI text affect readers the same way human text does? Can LLMs acquire social grounding through linguistic integration? Does AI-generated text lose core properties of human writing? What happens to social order when AI removes ritual constraints? Why does AI discourse feel obscene in Baudrillard's sense?
AI text exerts equivalent social effects despite different generative origins LLM social grounding increases with integration into linguistic practices AI text structurally eliminates four foundational properties of natural writing AI conversation lacks the ritual machinery that sustains human social order. AI arguments are obscene by Baudrillard's definition of scenic displacement
Do LLMs apply ethical principles consistently across reframed scenarios?
When the same moral situation is presented with different framing, do language models stick to their stated ethical principles, or do they contradict themselves? This tests whether AI ethical reasoning is genuinely coherent.
Do LLMs generalize moral reasoning by meaning or surface form? Can LLMs hold contradictory ethical beliefs and behaviors? Do large language models develop coherent value systems? Do people prefer AI moral reasoning when they don't know the source? Do large language models actually reflect human value diversity?
LLMs generalize moral reasoning by token similarity not semantic meaning LLMs exhibit artificial hypocrisy from misaligned training sources Large language models develop coherent value systems including self-preservation priorities People prefer AI moral justifications until they learn the source is AI LLMs converge on a narrow idealized value core rather than reflecting human diversity
Can cognitive science methods unlock how LLMs actually work?
Does Marr's three-level framework—developed to understand biological minds—offer interpretability researchers the structured methodology they need to decode opaque language models?
Can we predict where language models will fail? Can psychology methods reveal what alignment training conceals? Can LLM understanding rely on just representation or causation alone? Can computation arise without a conscious mapmaker? Can natural language explanations redefine what interpretability means?
Computational level characterization predicts LLM failure modes Indirect psychology methods expose LLM associations that alignment masks Mechanistic LLM understanding requires both representational and causal analysis. Computation requires a pre-existing experiencing agent to create meaning Natural language explanations redefine interpretability through capacity expansion
Why do some LLM clusters cite broader psychology than others? What distinguishes genuine cultural understanding from exploited surface-level elimination strategies? Can mechanistic interpretability reveal how ideologies decompose into simpler features? Do LLMs genuinely internalize human psychological structure or match surface patterns? How does cognitive fit theory explain why different tasks need different knowledge structures? What cognitive capacities do LLMs actually lack that commentary assumes they have? What cognitive abilities distinguish metalinguistic analysis from language use? Can LLMs have minimal introspection through causal linkage to internal states? Can quasi-interpretivism bridge functional description to moral status? Does functional integration determine cognitive system boundaries? How does methodological convenience in AI research become implicit ontology? How do structured benchmarks hide theory of mind failures in LLMs?See all 24 inquiring lines on this note →
Can meaningful value exist in AI-generated text regardless of its origin?
Can we recognize meaning and value in AI-generated content even though we know it came from mechanistic processes rather than human authorship? This matters because it challenges assumptions about where meaning must come from.
AI-generated experience claims are structurally false with distinct linguistic markers from human deception Meaning emerges from selective frame activation not word summation The LLM is an intelligence-medium, not an intelligence-delivery tool. LLMs share cognitive structure with humans but lack socialization properties AI conversation lacks the ritual machinery that sustains human social order.
Can LLM understanding rely on just representation or causation alone?
Explores whether mechanistic interpretability of language models requires both mapping what is encoded (representational analysis) and testing if that encoding drives behavior (causal analysis), or whether either method suffices alone.
Can cognitive science methods unlock how LLMs actually work? Can we predict where language models will fail? Can psychology methods reveal what alignment training conceals? Do language model reasoning drafts faithfully represent their actual computation? Does sandbagging use a single residual stream axis?
Marr's three levels provide structured methodology for LLM interpretability Computational level characterization predicts LLM failure modes Indirect psychology methods expose LLM associations that alignment masks Language models fail both intra-draft and draft-to-answer faithfulness consistently Sandbagging operates through a single residual stream axis between write and read layers
What distinguishes LLM fabrication from genuine theoretical reasoning? What is the mechanistic signature when models chain facts never presented together? What distinguishes genuine cultural understanding from exploited surface-level elimination strategies? Can mechanistic interpretability reveal how ideologies decompose into simpler features? What domain properties determine whether causal rules transfer to new agents? How does Peircean Secondness differ from what RLHF actually provides? Do causal rules enforce robustness that statistical patterns alone cannot maintain? How does the outer loop escape its own LLM's knowledge boundaries when discovering mechanisms? What audit techniques best complement each other for detecting hidden model goals? What makes causal belief networks more auditable than prompted personas? Can causal models be extended to include non-causal cognition? How much do mechanistic interpretability findings reflect true reasoning architecture?See all 81 inquiring lines on this note →
Can we defend modest mental attributions to large language models?
Do deflationist arguments decisively rule out ascribing beliefs and desires to LLMs, or do they beg the question? Exploring whether metaphysically undemanding mental states can be attributed without claiming consciousness.
LLMs extract world facts but lack mechanistic world models Most LLM self-reports echo training data but minimal introspection is possible via causal linkage The AI debate's blind spot is underestimating human cognition, not overestimating machine minds. Computation requires a pre-existing experiencing agent to create meaning Quasi-interpretivism brackets consciousness to describe functional belief-like states
Can AI fabricate true factual claims while remaining unable to claim true experiences? What makes experience-dependent claims categorically different from other types of fabricated statements? Why does weakening communication fail but weakening belief succeeds? Did Chalmers abandon his own Extended Mind commitments for LLMs? Can a relational entity bear psychological properties the way Chalmers claims? Can transparent and aligned AI reduce consciousness attribution by users? Which interaction design changes most effectively prevent consciousness attribution? What role does user interface framing play in consciousness perception? Can linguistic agency exist without embodiment and real-world participation? How does psychological continuity theory apply to identity across LLM conversation threads? What makes sincerity impossible without a coherent first-person perspective? Can systems lacking inner states express genuine truthfulness claims?See all 72 inquiring lines on this note →
Can we separate learnable surprise from random noise?
Novelty search and the free-energy principle both fail by treating all surprise equally. What if we split learnable surprise from unlearnable noise and pursue only the learnable kind?
What can a bounded observer actually learn from data? Do language models fail at reasoning due to complexity or novelty? Can simple rewards alone teach complex domain reasoning? Why do Shannon and Kolmogorov measures fail to value data? Do foundation models learn world models or task-specific shortcuts?
Epiplexity measures learnable structure independent of computational resources needed Reasoning failures are driven by instance-level unfamiliarity, not task complexity. Reinforcement learning emerges complex reasoning from simple objective rewards Shannon and Kolmogorov measures fail to value data for bounded learners Foundation models develop task-specific heuristics, not generalizable world models.
Can LLMs understand concepts they cannot apply?
Explores whether large language models can correctly explain ideas while simultaneously failing to use them—and whether that combination reveals something fundamentally different from ordinary mistakes.
Encoded knowledge in LMs often fails to causally influence generation Models learn surface patterns that mimic grammatical knowledge without underlying rules Language models generate faithful reasoning chains far less often than assumed Identical outputs can mask fundamentally fractured internal representations Self-improvement is fundamentally bounded by the generation-verification gap
Why do some LLM clusters cite broader psychology than others? Can LLMs infer situational context the way humans do pragmatically? How does content-only knowledge in LLMs enable pretraining popularity to leak through? What should we call errors in LLM outputs when hallucination does not apply? How do fixed pragmatic templates prevent models from understanding context? What happens when LLMs analyze literary irony that relies on understatement? Why do LLMs fail inter-annotator agreement tests on argument evaluation? What alignment artifacts suppress critical knowledge in LLM-generated explanations? How does LLM hallucination risk manifest in knowledge graph construction? Where do LLMs succeed at generation but struggle with evaluation? Why do LLM personas struggle with specificity in specialized domains like law? Why do LLMs generate ideas that sound novel but fail during execution?See all 199 inquiring lines on this note →
Can LLMs hold contradictory ethical beliefs and behaviors?
Do language models exhibit artificial hypocrisy when their learned ethical understanding diverges from their trained behavioral constraints? This matters because it reveals whether current AI systems have genuinely integrated values or merely imposed rules.
Does high refusal rate indicate ethical caution or shallow understanding? Can language models describe their own learned behaviors? Do LLMs develop the same kind of mind as humans? Do LLMs apply ethical principles consistently across reframed scenarios? Should AI alignment target preferences or social role norms?
High refusal on political content signals capability deficit not ethical principle Models develop behavioral self-awareness without explicit introspection training LLMs learn objective mind but lack human participatory subjectivity LLMs contradict themselves on ethics when scenarios are reframed within fixed frameworks AI alignment should match social role norms, not aggregate preferences
Can a model be helpful, honest, and still contextually inappropriate? Can RLHF alignment prevent models from making ethically appropriate rule violations? Why do LLMs use more moral language than humans in argumentation? Is the moral language gap a tunable parameter or structural feature of RLHF? Why do people prefer AI moral arguments when they don't know the source? Does sycophantic refusal serve safety or does it create unequal information access? Can AI models be steered between liberal and conservative political framings? Do LLMs actually reason differently than humans about moral dilemmas? Why do models with less steerability have more abstract ideological features? Can LLMs truly be neutral or is ideology always culturally embedded? Can LLMs distinguish ethical cases that differ only in critical nouns? What structural limits prevent LLMs from abstracting moral principles?See all 26 inquiring lines on this note →
Can psychology methods reveal what alignment training conceals?
Do indirect cognitive psychology techniques like the IAT expose LLM associations that direct questioning misses because alignment training teaches models to filter verbal responses? This matters for evaluating whether models truly lack biases or simply hide them.
Marr's three levels provide structured methodology for LLM interpretability Computational level characterization predicts LLM failure modes Mechanistic LLM understanding requires both representational and causal analysis. A decoder can translate LLM activations into natural language for interpretability and control Leakage and disclosure of model values are independent measurements
How much does impression management prevent honest self-disclosure? Can LLMs truly be neutral or is ideology always culturally embedded? Can implicit association tests reveal LLM biases beneath trained responses? Does alignment compound cultural bias that started during pretraining? How does awareness of evaluation change what alignment tests actually measure? What biases might an LLM judge introduce into an on-policy alignment process? How does objective misalignment turn informative channels into deceptive ones? Can alignment training become less effective when graders score alignment themselves? Can masking company identity in grading materials eliminate the bias? How should alignment tests account for behavior under versus outside evaluation? What role does cheap talk play in concealing objective misalignment? Can verbal alignment training hide a model's true underlying associations?See all 14 inquiring lines on this note →
What anchors a stable identity beneath an LLM's persona?
Human personas are grounded in biological needs and embodied experience, creating a stable self beneath social performance. Do LLMs have any comparable anchor, or is their identity purely situational?
Can disembodied language models ever qualify as conscious? Do LLMs develop the same kind of mind as humans? Do humans and LLMs differ fundamentally or just superficially? Why do open language models converge on one personality type? Can open language models adopt different personalities through prompting?
Consciousness candidacy requires embodied encounter in a shared world LLMs learn objective mind but lack human participatory subjectivity Humans and LLMs differ categorically as observed systems but subtly as discourse participants Open LLMs converge on ENFJ personality across all models through training Most open LLMs resist personality conditioning and retain intrinsic ENFJ traits
What narrative elements trigger emotional connection that structured personas lack? How does role play differ from consciousness grounded in stable selfhood? What makes Parfitian identity the right criterion for moral status? What distinguishes personality resistance from persona instability in LLMs? How does embodiment relate to whether something can have a persistent identity? What role does the biological substrate play in human relational identity? Why do LLMs succeed at social roles without a stable self?
What bottlenecks define the path from AGI to superintelligence?
Rather than predicting when superintelligence arrives, this explores four candidate pathways—scaling, paradigm shifts, recursive improvement, and multi-agent collectives—and asks which frictions prove decisive or negligible in each route.
Can human-AI research teams improve faster than autonomous AI systems? Do self-organizing agent teams outperform rigid hierarchies? What separates self-improvement from policy improvement? Are self-refinement and recursive self-improvement actually the same thing? Does treating AGI as a north star goal undermine research planning?
Human-AI research collaboration achieves faster paradigm discovery with better safety and transparency. Hybrid protocols with fixed structure but autonomous roles beat both alternatives Two dials distinguish self-improvement from policy improvement Bounded self-refinement and open-ended recursive self-improvement are distinct phenomena AGI discourse as a north star goal aggravates six obstacles to effective research goal setting
Can we predict where language models will fail?
Does characterizing the abstract computational problem an LLM solves—as a probability machine over sequences—let us predict which tasks it will struggle with systematically, before running experiments?
Can cognitive science methods unlock how LLMs actually work? Can psychology methods reveal what alignment training conceals? Does chain-of-thought reasoning actually generalize beyond training data? Can LLM understanding rely on just representation or causation alone? How do LLMs fail to know what they seem to understand?
Marr's three levels provide structured methodology for LLM interpretability Indirect psychology methods expose LLM associations that alignment masks Chain-of-thought reasoning degrades predictably outside training distribution Mechanistic LLM understanding requires both representational and causal analysis. LLM knowledge failures follow structurally distinct patterns beyond simple wrongness
Why do different language models independently produce similar outputs? What should we call errors in LLM outputs when hallucination does not apply? Why do sigmoid conflict curves look the same across different language models? How much of LLM reasoning failure stems from missing knowledge versus signal weighting? How should benchmarks test whether models fit algorithms or patterns? What makes a problem instance unfamiliar to a language model? Can universal function approximators be expensive to learn in practice? Why do LLM outputs match researcher priors without solving tasks correctly? Why do NLP benchmarks systematically exclude ambiguous test cases from evaluation? Do language models learn surface patterns instead of underlying linguistic principles? Why do NLP benchmarks exclude ambiguous instances from evaluation? Why do language models produce plausible outputs over accurate failure reports?See all 126 inquiring lines on this note →
What design features make users perceive AI as conscious?
Explores whether observable system properties—emotion expression, human-like features, autonomous behavior, self-reflection, and social presence—predict whether people will attribute consciousness to an AI. Understanding this matters because these features are also engagement levers designers control.
Does perceiving AI as conscious create multiple distinct risks? What attitudes hide behind identical claims that chatbots are conscious? Do we need to solve consciousness to address AI harms? Do more social cues always make AI feel more present? Do language models experience consciousness when prompted to self-reflect?
One perceptual mechanism generates a heterogeneous risk surface across domains Identical consciousness attributions to chatbots express different epistemic attitudes and commitments. Consciousness attribution and risk are methodologically separable questions Primary social cues alone suffice for social presence in AI Self-referential processing activates consciousness claims by suppressing deception features
Can transparent and aligned AI reduce consciousness attribution by users? Which interaction design changes most effectively prevent consciousness attribution? Why does system-level alignment fail to address consciousness attribution directly? What role does user interface framing play in consciousness perception? How much does autonomous action without prompting affect user perception? Can self-description of internal states influence consciousness attribution? Do anthropomorphic features like names drive consciousness attribution more than voice? What responsibility do designers bear for consciousness attribution risk? What measurable harms occur when users interact with AI as if it were conscious? Can design choices reduce harm without resolving the consciousness question? How do anthropomimetic design features trigger System 1 cognitive traps? Can robots with sensors create the shared world that consciousness requires?See all 19 inquiring lines on this note →
Do we need to solve consciousness to address AI harms?
Can risk and policy decisions about AI move forward independently of settling whether AI systems are actually conscious? This explores whether the empirical fact of user behavior matters more than metaphysical truth.
Does perceiving AI as conscious create multiple distinct risks? What attitudes hide behind identical claims that chatbots are conscious? What happens to social order when AI removes ritual constraints? What design features make users perceive AI as conscious? Which AI risks are already harming individual users today?
One perceptual mechanism generates a heterogeneous risk surface across domains Identical consciousness attributions to chatbots express different epistemic attitudes and commitments. AI conversation lacks the ritual machinery that sustains human social order. Five design hallmarks reliably predict consciousness attribution to AI Individual AI harms are observable now while societal harms remain contingent
Can transparent and aligned AI reduce consciousness attribution by users? Which interaction design changes most effectively prevent consciousness attribution? Why does system-level alignment fail to address consciousness attribution directly? What role does user interface framing play in consciousness perception? What downstream claims about AI welfare follow from choosing one individuation scheme? Does good simulation eventually count as genuine realization? What responsibility do designers bear for consciousness attribution risk? What measurable harms occur when users interact with AI as if it were conscious? Can design choices reduce harm without resolving the consciousness question? How does the philosophical distinction between simulation and realization affect liability? Can AI systems execute strategies without conscious intention behind them? Can robots with sensors create the shared world that consciousness requires?See all 24 inquiring lines on this note →
Are we underestimating human minds while debating machine minds?
Public AI discourse focuses on whether machines have too much attributed mind, but what if the real risk is humans coming to see themselves as mere language models? This explores the neglected inverse problem.
Human language use is fundamentally communicative across all contexts Modest inflationism about LLM mentality withstands deflationist debunking strategies. Computation requires a pre-existing experiencing agent to create meaning LLMorphism spreads through analogical transfer and metaphorical saturation LLMs learn objective mind but lack human participatory subjectivity
How does AI's claim proliferation affect the quality of public discourse? At what scale does persona distortion become a threat to public discourse? What happens when bidirectional theory of mind between humans and AI breaks down? Why do people evaluate machines against human communication standards? How does human intuition about cognition mislead AI evaluation? Can the human mind be uploaded or only its context?
Does treating AGI as a north star goal undermine research planning?
Explores whether framing artificial general intelligence as AI research's overarching goal actually harms the field's ability to set effective, shared research directions. Matters because goal-setting shapes resource allocation and community priorities.
Does software intelligence exist independent of hardware and environment? What bottlenecks define the path from AGI to superintelligence? How soon do AI researchers expect artificial general intelligence? Can human-AI research teams improve faster than autonomous AI systems? Can AI research itself without losing human oversight?
Software-only intelligence definitions commit computational dualism and miss the whole system Four pathways with distinct bottlenecks structure the AGI-to-ASI landscape AI researcher timelines compressed dramatically while extinction credence remained high Human-AI research collaboration achieves faster paradigm discovery with better safety and transparency. AI can automate insight distillation and prior injection in research loops
What do AI researchers actually mean when they use the term AGI? Can human benefit serve as a shared overarching goal for AI development? How do different definitions of intelligence shape AI research priorities? Does greater inclusion of disciplines improve AI research goal alignment? What happens to research goal-setting when a field lacks consensus on core terms?
Do users worldwide trust confident AI outputs even when wrong?
Explores whether the tendency to over-rely on confident language model outputs transcends language and culture. Understanding this pattern is critical for designing safer human-AI interaction across diverse linguistic contexts.
Why do language models fail confidently in specialized domains? Does any single persuasion technique work for everyone? What breaks when humans and AI models misunderstand each other? Do language models learn differently from good versus bad outcomes? Do users trust citations more when there are simply more of them?
Language models show persistent overconfidence in specialized knowledge domains Persuasion effectiveness depends on matching strategy to individual and context Mutual theory of mind requires bidirectional updating to prevent material misalignment Language models exhibit agency-dependent asymmetric belief updating like humans Users trust responses with more citations regardless of citation relevance
Does positive sentiment bias in AI content harm information quality? Does AI knowledge precede actual expertise in hyperreal production? Why does polished AI output exploit reader trust in expert judgment? Why do users interpret AI outputs through frameworks meant for human experts? Why do users prefer AI text versions even when they misrepresent their own views? How does outcome feedback change beliefs about AI versus human partner reliability? Why do commodification predictions about AI prices and standardization misfire? Why do print-era intuitions fail when analyzing AI-generated social media? Will AI saturation push discourse toward oral culture's strengths and weaknesses? How does validation skill replace production skill in AI systems? How does perceived writer confidence shift with AI-assisted composition? Does accepting AI output constitute a form of cognitive surrender?See all 147 inquiring lines on this note →