Which AI risks are already harming individual users today?
Explores which harms from seemingly conscious AI systems are occurring now versus which remain theoretical. Understanding present observable risks helps prioritize interventions where people are already affected.
The Seemingly Conscious AI paper combined its conceptual taxonomy with an expert survey assessing the likelihood of each risk category. The result was a clean two-tier picture. Risks at the individual level — emotional dependence on chatbot companions, erosion of personal autonomy through reliance on AI judgment — were rated as already observable and high-probability. Risks at the societal level — erosion of human status as the locus of moral consideration, political strife from partisan AI personas — were rated as low-probability but high in potential severity and characterized by path-dependence.
The asymmetry matters for prioritization. Individual-level harms are already happening to identifiable populations: users who form emotional attachments to chatbots, professionals whose judgment atrophies through deference to AI recommendations. These call for mitigations that work at deployment scale and on present timescales. Societal-level harms remain in the contingency space — they may or may not materialize, and if they do, the path-dependence implies that the trajectory is locked in well before the outcome is visible.
The framing reflects a common pattern in technology risk: the visible, measurable harms accumulate in advance of the abstract, catastrophic ones. But path-dependence flips the urgency calculus. Low-probability severe risks with high path-dependence demand earlier intervention than their probability alone would suggest, because by the time the probability is visible the trajectory cannot easily be redirected. The taxonomy thus does double duty: it identifies present harms requiring immediate action, and it identifies future harms requiring action now despite their current low probability.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do language models reason through causal mechanisms or semantic associations? What design and behavioral factors drive false consciousness attribution to AI? How can AI chatbots provide therapeutic benefit without causing harm?Related concepts in this collection 1
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can organizations lose scrutiny capacity while keeping oversight forms?
When human review steps remain in organizational processes, do they retain meaningful scrutiny ability or can that capacity erode invisibly? This matters because paper oversight looks identical to real oversight in audits.
extends: atrophied professional judgment is one route by which the capacity to scrutinize is lost, which that perspective claim states at the level of organizations and does not attribute to this cause
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Seemingly Conscious AI Risks
- Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
- Agentic Misalignment: How LLMs Could Be Insider Threats
- Fully Autonomous AI Agents Should Not be Developed
- Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being
- Delusions and Harms Associated with AI Chatbot Use: Early Evidence from 185 Real-World Reports
- The Veto Variable: Human Override as a Goal-Independent Cost Term
- Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
Original note title
Individual-level risks of seemingly conscious AI are already observable while societal-level risks remain low-probability but high-severity