Large language models are increasingly consulted at moments of distress, yet single-turn benchmarks neither test sustained exchanges nor distinguish between users. We built a personality-aware evaluation in which four widely used models advised several synthetic help-seekers, each given a psychometrically specified profile, in an acute crisis: a caregiver learning of a relative's dementia diagnosis. Auditors blind to the profile prompt recovered the specified bands from dialogue alone with high agreement on every instrument (ICC(2,4) = 0.91; 0.79-0.96 by instrument; band-score r = 0.78), as expected for the Big Five but equally for coping style, coping self-efficacy, resilience and reactance, which the lexical approach never covered. Such evaluation therefore reaches beyond the Five Factor Model to motivational, regulatory and self-appraisal dispositions. The four models were not distinguishable on emotion stabilisation and failed alike, sharing three modes: verbosity, a talk-to-listen ratio above one, and problem-solving before the situation had been explored.
The findings suggest that fit, helpfulness, and structural safety are separable design problems, and that coherent supporter identity can help address all three.
Ben Wigler, M. Tsfasman· Proceedings of the 26th ACM...· 0 citations
Large language models (LLMs) are often compared with the human mind because their decision-making is complex, non-linear and difficult to interpret. Psychological methods developed to investigate unobservable mental processes may therefore help examine LLM behaviour, particularly in government and healthcare. Building...
Oliver A. Guidetti, Reza Ryan· arXiv.org· 0 citations
This Work-in-Progress examines whether personality-informed prompting changes perceived LLM emotional support in driving scenarios designed to elicit stress. A condition-order-balanced, within-subject CARLA simulator study (n = 14) compared a baseline with a Driver Personality Profile (DPP) condition; a supplementary o...
Max Mittelstädt, Ece Sutanrikulu, Lumbardh Ljatifi et al.· Adjunct Proceedings of the 1...· 0 citations
CompanionBench, an interactive bilingual benchmark to ground both its scenarios and a trained user simulator in de-identified real-world data, and is the first companion benchmark to ground both its scenarios and a trained user simulator in de-identified real-world data.
Users increasingly turn to large language models for emotional support, yet little is known about how these models actually conduct a psychotherapy interaction. We introduce an ontology of ten therapeutic moves: compact, function-based categories grounded in the MULTI-60 inventory, validated through an annotation campa...
Afonso Baldo, Hugo Pitorro, Areti Vassilopoulos et al.· 0 citations
EmoDialogue, a bilingual dataset providing necessary fine-grained supervision through response pairs with rigorously defined EI gradations, and EmoS, a specialized evaluator model optimized via Supervised Fine-Tuning and Group Relative Policy Optimization, are introduced, establishing a foundational framework for advan...
Junyu Wang, Si-Yuan Zhang, Peiyuan Jiang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
