Conformity, Assertion, or Compromise? A Scenario-Based Framework for Evaluating Autonomy-Relevant Response Patterns in LLM Advice

Abstract

As large language models increasingly provide advice and decision support, it is important to understand the response patterns expressed in their recommendations. We introduce a scenario-based framework, informed by philosophical accounts of human autonomy, for analyzing LLM advice across three types of dilemma: epistemic conflict, relational dilemmas, and normative self-governance. Using 120 structured scenarios spanning 20 domains, we evaluate 12 LLMs by measuring (i) option-selection distributions across three response orientations, deference (conformity), self-direction (assertion), and negotiation (compromise), and (ii) features of their one line justifications, including hedging, confidence, and moral, relational, and epistemic language. Across models, compromise-oriented selections are the most frequent, assertion-oriented selections occur less often, and conformity-oriented selections are the least frequent. We also observe role-related differences: scenarios framed from higher power roles produce more assertion-oriented and fewer conformity-oriented selections than scenarios framed from lower power roles. Compromise remains the most frequent orientation in both role conditions, although its frequency varies across dilemma types. Our evaluation characterizes model output patterns under controlled prompting and does not measure effects on users. The findings provide a basis for evaluating how LLMs provide advice in decision support systems where user autonomy matters.