Can structured prompting improve cognitive distortion detection?
This explores whether breaking distortion diagnosis into discrete stages—mirroring clinical CBT workflow—helps language models identify and classify thinking patterns more accurately than standard approaches.
Diagnosis of Thought (DoT) prompting structures cognitive distortion detection into three stages that mirror how clinical psychologists actually diagnose thinking patterns:
Stage 1 — Subjectivity Assessment. Patient speech mixes reality (objective facts) with interpretations (subjective thoughts). The first step separates these, summarizing objective facts into "situations" that serve as the evidence base for diagnosing the subjective thoughts. This prevents the model from treating interpretations as facts.
Stage 2 — Contrastive Reasoning. Based on the situation, the model generates reasoning processes both supporting and contradicting the patient's thoughts. By contrasting two different interpretations grounded in the same facts, distorted thought patterns become visible. This mirrors the CBT technique of examining evidence for and against a belief.
Stage 3 — Schema Analysis. The model identifies the underlying cognitive structures (schemas) that produced the specific reasoning process, mapping them to recognized cognitive distortion types (emotional reasoning, overgeneralization, mental filter, should statements, all-or-nothing, mind reading, fortune telling, magnification, personalization, labeling).
DoT achieves >10% relative improvement on distortion assessment and >15% on classification over ChatGPT zero-shot. Expert evaluation rated the generated rationales as "comprehensive" or "partially good" at high rates. The three-stage structure generates explanations that are clinically useful — therapists could use them as starting points for case formulation.
Since Can breaking down visual reasoning into three stages improve model performance?, structured multi-stage prompting that maps to established cognitive frameworks consistently outperforms unstructured approaches. DoT is the CBT-specific instance of this general principle: domain-expert reasoning workflows decompose into inspectable stages that LLMs can follow.
Inquiring lines that read this note 24
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Does warmth and empathy training systematically degrade model reliability? Do language models lack essential therapeutic presence and engagement?- Can models succeed at mental health tasks without integrating multiple psychological traditions?
- What other therapy constructs could be measured from transcripts using this approach?
- How do structured cognitive models prevent repetitive and contradictory patient dialogue?
- What makes Beck's diagram effective for constraining simulated patient behavior?
- Can large language models actually deliver cognitive behavioral therapy techniques?
- Do problem-solving defaults in LLM therapists actually undermine therapeutic effectiveness?
- Do worksheet-based structured formats work as well as embodied agents for therapy?
- What makes clinical theory grounding more effective than pattern matching alone?
- Why do Llama models struggle with cognitively distorted user expressions in therapy?
- Can AI provide therapy without challenging users to confront cognitive distortions?
- Does prompting or added context help LLMs understand therapeutic timing and depth?
- How do discourse-level patterns reveal cognitive distortions better than individual statements?
- Can the three-stage DoT framework detect all cognitive distortion types reliably?
- How do cognitive distortions appear as patterns in how people structure sentences?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can breaking down visual reasoning into three stages improve model performance?
This explores whether structuring visual reasoning through perception, situation, and norm stages—grounded in cognitive science—helps language models reason about socially complex scenes better than flat chain-of-thought approaches.
same structural insight applied to visual reasoning
-
Why do discourse patterns predict anxiety better than single words?
Explores whether anxiety detection requires understanding how statements relate to each other rather than analyzing individual words. This matters because it reveals what computational methods need to capture cognitive distortions.
distortions are discourse-level phenomena; DoT's contrastive reasoning stage addresses this
-
Can structured argument prompts make LLM reasoning more rigorous?
Does requiring language models to explicitly check warrants, backing, and rebuttals—rather than reasoning freely—improve reasoning quality and catch failures that standard step-by-step prompting misses?
parallel: domain-specific structured prompting improves performance
-
How do readers track segments, purposes, and salience together?
Can discourse processing actually happen in parallel rather than sequentially? This matters because understanding how readers coordinate multiple layers of meaning at once reveals where AI systems break down in comprehension.
DoT's three-stage decomposition aligns with Grosz and Sidner's three-component model: separating linguistic structure (what was said) from intentional structure (why it was believed) from attentional structure (what is salient) mirrors the subjectivity/contrastive/schema stages
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting
- LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices
- Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
- Detecting Cognitive Distortions from Patient-Therapist Interactions
- Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
- Eliciting Reasoning in Language Models with Cognitive Tools
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Cognitive Chain-of-Thought: Structured Multimodal Reasoning about Social Situations
Original note title
cognitive distortion detection benefits from structured three-stage prompting that separates subjectivity assessment from contrastive reasoning from schema analysis