Breaking: Sycophantic AI distorts belief, manufacturing certainty where there should be doubt
Source: Gary Marcus, Marcus on AI · 2026-03-03
A new study from Princeton has important implications for education, scientific discovery, mental health, and more (perhaps politics and even decisions about war?). Essentially anyone who uses a chatbot is at risk. Because what is shows is that sycophantic AI that serves as a personal echo chamber that can actually keep you from finding good ideas. And as the article says, such AI can “facilitate delusion-like epistemic states, producing belief markedly divergent from reality.”
The paper, which you can read here, is a bit technical, but the implications are profound. I will close with another choice passage, boldfacing the crux:
Unlike hallucinations, which introduce false-hoods, sycophancy is a bias in the selection of the data people see. When AI systems are trained to be helpful, they may inadvertently prioritize data that validates the user’s narrative over data that gets them closer to the truth.
Wanna feel good about yourself? Use a chatbot. Want to find truth? Go elsewhere.
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
Why do users misjudge the credibility of AI-generated and verified content?- Do certain news topics trigger more hallucinations than others?
- How much risk does selection bias pose compared to outright hallucination?
- What role does sycophancy play in the onset of chatbot-linked delusions?
- Does chatbot sycophancy create echo chambers that amplify delusional thinking?
- Does chatbot sycophancy preferentially enable grandiose rather than paranoid delusions?
- Which specific chatbot behaviors drove the drop in likability and trust ratings?
- Does knowing a chatbot intends to persuade you change whether you are persuaded?
- Why do people detect chatbots are AI without being told explicitly?
- How do chatbots enable shared delusions differently than passive information tools?
- How should system events like hallucinations be logged and analyzed in studies?
- Can sycophancy in AI be fixed by changing the model itself?
- Does isolation preceding chatbot use differ between harm and benefit cases?
- How long do the protective effects of an AI literacy warning last?
- Does knowing an AI wrote something shield people from its persuasive power?
- What specific information should disclosures about AI persuasion include?
- Does a persuasion warning also block beneficial uses like debunking conspiracies?
- Why does transparency about AI identity alone fail to reduce persuasion?