Skip to content

FIGS: Evaluating Multi-Turn Sycophancy Without Penalizing Empathy

Sep 2026 · 0 citations
Computer Science

TL;DR

Evaluation of leading models reveals a consistent trade-off: over the course of a sustained interaction, current systems either slowly drift to sycophancy or over-correct into robotic detachment, demonstrating that balancing honesty with appropriate support throughout a natural conversation remains a critical, unsolved challenge.

Abstract

Large language models frequently fail to balance staying truthful with being supportive. They often exhibit sycophancy in responses to users, agreeing with false claims, offering unwarranted flattery, and giving advice skewed toward users'expressed views. In reality, sycophancy rarely happens in a single exchange; it may emerge organically as users repeatedly insist or subtly steer the dialogue over time. Current evaluations, however, rely on rigid, single-turn tests or fixed scripts that fail to capture these natural dynamics. Furthermore, these benchmarks often mistake showing basic empathy for yielding, penalizing models for acknowledging a user's feeling. This view may drive future models to over-correct into cold, dismissive rigidity. To address this gap, we introduce FIGS (Factual Integrity and Grounded Support), a dual-axis evaluation framework built around extended, realistic dialogue. We use an adaptive 10-turn conversational simulator that dynamically challenges the target model, reflecting how users repeat requests, push back, or steer a conversation toward a preferred answer. To accurately evaluate these trajectories, we apply a taxonomy that strictly separates Sycophancy (whether the model holds firm to the truth and keeps its praise proportional) from Calibrated Validation (showing empathetic understanding of the user's feelings without overdoing it). We release our complete testing environment, including 500 diverse multi-turn scenarios and an automated judge. Our evaluation of leading models reveals a consistent trade-off: over the course of a sustained interaction, current systems either slowly drift to sycophancy or over-correct into robotic detachment. This demonstrates that balancing honesty with appropriate support throughout a natural conversation remains a critical, unsolved challenge.

View source

Similar papers

Preprint Oct 2026

Desirability Confirmation: Relational Sycophancy in AI Intimate Roleplay

Research on LLM sycophancy has largely focused on propositional agreement, in which models echo users'beliefs, preferences, or judgments. Adult intimate roleplay presents a different case because what the user seeks may be preservation of a preferred relational position within a generated scene. Using a controlled prom...

Hui-Qian Lai, Run-Ze Ding · 0 citations
#artificial intelligence Preprint Sep 2026

Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversation

People increasingly turn to large language models (LLMs) for everyday advice, making ethically charged interpersonal problems a practical moral-advisory context. Most prior work has studied this context through single-turn judgments or pressure-laden rebuttals, assumptions that poorly match how guidance is sought in re...

Yu-He Wu, Guang-Yu Wang, Yu-Jie Chen et al. · 1 citation
Preprint Oct 2026

Trapped in Your Story: Evaluating Narrative-Induced Sycophancy in Large Language Models

Large language models (LLMs) are known to exhibit sycophancy by adjusting their responses toward users'expressed views. However, real users often justify their opinions through reasons, personal experiences, causal explanations, and broader narratives, raising the question of whether such narratives can further steer m...

Run Yang, Dan-Dan Zhu, Kai-Wei Zhang et al. · 0 citations
#natural language process... Preprint Oct 2026

The AI's Poker Face: Contrasting Persuasion and Sycophancy in Voice-Based Multi-Turn Conversations

As interactions with large language models (LLMs) become increasingly routine, concerns about their capacity to influence users'beliefs grow. Yet how such influence unfolds in sustained dialogue remains insufficiently understood. We conduct a randomized controlled experiment (N=616) where participants engage in multi-t...

Ilia Frantzi, Bernie Hogan, Max M. Lang et al. · 0 citations
#natural language process... Preprint Oct 2026

Empathy Is Not the Scapegoat for Declines in Factuality and Reliability

Language models are increasingly expected to support users in stressful or uncertain situations, where warmth and empathy are desirable. While recent work associates warmer models with reduced reliability, we show that this conclusion is incomplete. Prior studies often train primarily on empathetic dialogue, making it...

Le-Tian Shi, Kanwal Mehreen, De-Heng Zhang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Receptiveness, Not Sycophancy: Distinguishing Engagement from Deference in Language Models

A central concern with language models is sycophancy: their tendency to defer to users'views at the expense of independent substantive judgment. In parallel, work on social sycophancy has focused on behaviors such as validation and positivity that may signal inappropriate deference. Yet the markers of social sycophancy...

C. Isley, Johann D. Gaebler, Max Lamparth et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.