Results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.
Abstract
Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors. However, out-of-the-box LLMs respond fluently and empathetically, yet collapse into validation&reflection, regardless of what the user actually needs. They know theoretical CBT (scoring up to 96% accuracy on licensing exam questions) but fail to apply it effectively. We explore this gap with a knowledge-guided framework that treats CBT dialogue as controlled affective reasoning: user narratives are decomposed into Beck's Cognitive Conceptualization structure, grounded in clinical SNOMED CT concepts validated via Natural Language Inference, and a Multiple Chain-of-Thought (MCoT) strategy selection between Validation&Reflection, Socratic Questioning, or Alternative Perspectives. To measure whether such guidance actually changes behavior, we introduce the Protocol Leverage Force (F), a behavior-level metric that captures how far an intervention shifts a model away from its default response. Across three open-weight LLMs and 14 RealCBT-derived case studies, evaluated with human experts, valence-arousal trajectories, and linguistic entrainment, F shows that simply introducing protocol definitions via single chain-of-thought prompting fails to change LLM behavior, while MCoT on these definitions guides strategy selection better. Still, the effect stays within 1% (approx. 1.2-1.3%), and all models remain biased toward Validation&Reflection. These results show CBT knowledge alone does not ensure effective application, giving the affective-computing community instrumentation to measure where LLMs fall short.
Cognivia is an evidence-based artificial intelligence therapist that integrates automatic cognitive distortion identification and rational response generation and is proposed the first hierarchical quality evaluation framework for assessing LLM-generated rational responses, developed through collaboration between AI researchers and behavioral science experts.
Qi Chen, Siria Xiyueyao Luo, Jian Wang et al.· 0 citations
Users increasingly turn to large language models for emotional support, yet little is known about how these models actually conduct a psychotherapy interaction. We introduce an ontology of ten therapeutic moves: compact, function-based categories grounded in the MULTI-60 inventory, validated through an annotation campaign with five licensed psychologists, and scaled with a judge-based approach that matches expert agreement. Applying it to real counseling transcripts and model-led sessions, we compare the move distributions between human clinicians and a panel of frontier models. Models over-use inquiry at up to three times the human rate, neglect psychoeducation, and are strongly context-anchored: they carry forward strategies initiated by a human clinician but rarely initiate them themselves. Exposing the ontology as a set of tools roughly halves the mean deviation from the human move distribution and improves turn-level alignment with human therapist by 7-9 percentage points, without any fine-tuning.
Afonso Baldo, Hugo Pitorro, Areti Vassilopoulos et al.· 0 citations
Cognitive behavioral therapy (CBT) is a well-established psychotherapeutic approach, yet its delivery through AI systems remains challenging: small language models (SLMs) struggle with multi-step therapeutic reasoning, while large language models (LLMs) are often impractical for resource-constrained deployment. We address this gap by formulating CBT-based cognitive reframing as a task-oriented dialogue (TOD) problem with dialogue state tracking (DST), which exposes intermediate belief states—such as event, thought, and cognitive distortion—making the reasoning process explainable and auditable for clinical oversight. To support controlled experimentation, we introduce CREPAN, a synthetic dataset of over 6,800 expertguided CBT reframing dialogues for panic disorder, designed as a controlled experimental setting to investigate whether models can learn the structured reasoning prescribed by the CBT ABC model. We further propose Sequential Self-feeding Slot Prediction (S3), a lightweight reasoning scaffold that guides models through interpretable cognitive stages in a theoretically grounded order derived from Ellis’s ABC model. Experiments show that S3-T5 attains a JGA of 0.7781, narrowing the gap with 8B-scale LLMs, with the sequential self-feeding structure enabling interpretable step-by-step reasoning. These results suggest that cognitive-theoretic structure is central to robust therapeutic state tracking, offering a practical path toward lightweight, interpretable, and clinically accountable AI systems for CBT-based counseling1.
Subin Kim, Hoonrae Kim, G. G. Lee· ACM Transactions on Computin...· 0 citations
Synthetic generation of Cognitive Behavioral Therapy (CBT) sessions is challenged by two competing demands: adhering to strict therapeutic structure while modeling the resistant, unpredictable behavior of real patients. Existing script-based methods fail to capture dynamic therapeutic interactions, while multi-agent approaches struggle to adhere to CBT's sequential structure; both suffer from sycophancy, producing overly compliant patients that misrepresent real clinical settings. In this work we introduce ODRA, a novel framework for synthesizing therapy dialogues through a Chain-of-Thought (CoT) strategy grounded in foundational CBT guidelines (Beck, 2020). ODRA further incorporates a resistance orchestrator to solve patient sycophancy, which employs steering techniques to elicit behaviors aligned with their resistance level. Automated and expert evaluations show that ODRA significantly outperforms existing methods across therapeutic skills, CBT alignment, and patient behavioral fidelity, with licensed psychologists preferring ODRA sessions across 12 of 13 clinical metrics. Furthermore, models fine-tuned on our dataset demonstrate superior therapeutic performance against both cooperative and resistant patients, validating that explicit resistance modeling in synthetic training data directly translates to downstream clinical robustness.
Javier Rodríguez-Juan, Hiba Arnaout, Jose Garcia-Rodriguez et al.· 0 citations
Self-criticism is a pervasive cognitive pattern that undermines mental health and academic functioning in university students. Current VR-based mental health interventions focus predominantly on anxiety disorders through exposure paradigms, relying on fixed, pre-authored content— an approach that places limited emphasis on cognitive restructuring, which must target each individual’s specific maladaptive thought patterns. In this paper we present a VR system that enables participants to engage in cognitive restructuring of self-critical personalized thoughts, developed through interdisciplinary collaboration between computer science and clinical psychology. The system pairs a web-based clinical interface—where psychologists configure patient-specific self-critical phrases and compassionate alternatives—with an immersive Unity application for Meta Quest 2, in which users confront their own thoughts through the embodied metaphor of a broken mirror. We describe the system architecture, interaction design, and design rationale based on principles from Cognitive–Behavioural Therapy and Compassion-Focused Therapy. We discuss design implications and outline directions for a planned empirical evaluation with university students
Eva Oliveira, Diogo Martins, N. Rodrigues et al.· International Conference on...· 0 citations
As artificial intelligence (AI) becomes increasingly integrated into educational contexts, understanding how learners engage with AI-generated feedback is critical. Although prior research has examined the effectiveness of automated writing evaluation, less attention has been given to the cognitive and affective processes through which learners interpret and respond to such feedback. This study employed a constructivist grounded theory approach to investigate how Chinese undergraduate EFL learners engaged with AI-supported feedback during L2 writing revision. Participants were 24 English-major undergraduates at a public university in China. Data were generated through six semi-structured small-group interviews and analyzed through iterative open, axial, and selective coding. The analysis identified three interrelated domains—cognitive engagement, contextual evaluation, and affective regulation—which were integrated into an interpretive cognitive–affective dual-path model. When feedback was perceived as transparent, manageable, meaning-preserving, and aligned with task demands, learners demonstrated stronger cognitive engagement, greater willingness to experiment with complex structures, and sustained motivation. In contrast, feedback perceived as opaque, overly evaluative, or misaligned with learner expectations was associated with negative affect, selective uptake, risk avoidance, and strategic simplification. Learner evaluative filtering appeared to shape how noticing developed into revision action. These findings provide a process-oriented account of learner engagement in AI-supported writing environments and suggest that the influence of AI-generated feedback is filtered through learners' evaluative judgement, contextual appraisal, and affective regulation.
Ting Li, Azlina Binti Abdul Aziz, Nur Ainil Sulaiman· Frontiers in Psychology· 0 citations