MATE: Diagnosing Empathy Calibration Failures in Multi-Turn Human-LLM Interaction

Abstract

Large language models are increasingly used in emotionally consequential interactions. Response-level evaluation, however, struggles to diagnose how empathy fails across turns. We introduce MATE (Multi-turn Assessment of calibraTed Empathy), a framework that evaluates empathy as a multi-turn, perception-centered process. Across two controlled studies (N=82), baseline responses exhibit disclosure-insensitive miscalibration: generic validation and formulaic tone read as hollow across turns. A prompt-level self-critique condition makes these patterns less prominent but coincides with a different failure, agreement-skewed miscalibration, where affirmation becomes insufficiently contingent on disclosure context under high persona alignment (62.5% of fit participants in Study 2). Response-level analysis is consistent with a tone-agreement dissociation, as the condition shifts tone toward naturalness while agreement density also increases (+98% under fit vs. +25% under unfit). These findings reframe empathic behavior as relational calibration-consistency among disclosure depth, persona, and response behavior-rather than maximization. Response behavior itself spans tone, agreement, and contextual specificity, sub-dimensions that can drift independently.