Sep 2026· AI and Ethics· Vol 6· 0 citations· 30 references
Psychology of Moral and Emotional Judgment
Abstract
Evaluating whether large language models (LLMs) reason about morality in human-like ways requires more than measuring whether they produce the right outputs in isolated cases. Existing approaches – including scalar agreement, distributional analysis, rationale classification, and consistency testing – compare model and human responses one case at a time and cannot detect how a model structures its moral judgments. This paper applies and extends Peterson and Gärdenfors's (2024) geometric moral space framework to evaluate structural value alignment in LLMs, that is, whether the relational structure reconstructed from a model's responses matches the structure reconstructed from human respondents on the same task. Sixteen LLMs across three scale tiers (small, medium, and large) were evaluated on ten moral cases under both zero-shot and chain-of-thought (CoT) prompting. The results reveal phenomena invisible to case-by-case metrics. First, all five large models achieve full geometric alignment on these cases (moral space overlap, MSO = 1.00) in at least one prompting condition, and three do so in both; two medium models also reach MSO = 1.00. Second, structural alignment is bimodal in the sense that models either replicate the human moral space or fall substantially short, with no intermediate cases. Third, smaller models systematically lack a distinct fairness region, a failure of moral categorization rather than of labeling. Fourth, CoT prompting simultaneously improves case-by-case similarity correlations and worsens geometric alignment in smaller models, showing that the two can move in opposite directions. Because structural alignment is measured from model responses, it does not by itself establish that a model holds the values in question. It is, however, a more demanding test than case-by-case agreement, and a better motivated and more informative basis for value alignment measurement.
This study empirically confirms the presence of LLM values, accurately quantifies their shifts, and achieves more efficient and precise steering than conventional blind training, all without degrading general capabilities.
Kelvin Zhang, Jing-Yu-Gin Chen, Yu-Fan Liu et al.· 0 citations
This work evaluates LLM politeness judgments using two English-language datasets with complementary annotation formats, finding that inter-model agreement is stronger than model--human agreement.
Rong Wang, Kun Sun, Ya-Dong Guo· Conference on Empirical Meth...· 0 citations
The results show that pluralistic prompting is a practical lever for value-conditional behavior and a fertile instrument for creative AI exploration in large language models with ten-principle constitutions.
Arjun Balaji, Aarushi Nema, Neha Kamath et al.· 0 citations
Large language models (LLMs) are used to simulate human participants in psychological research. We asked whether LLMs that reproduce human evaluations of a whistleblower's moral character also reproduce the motive attributions that accompany them. Five LLMs and two human samples (N = 125 and N = 742) evaluated a physic...
This work trains six independent linear probes on open-weight language models, one per Moral Foundations Theory (MFT) category, and examines how the resulting directions relate to each other in representation space, finding the directions neither collapse into a single moral detector nor isolate from one another.
Large Language Models (LLMs) are entering democratic contexts as instruments of governance, where the challenges at hand are ill-structured, marked by ambiguity and contestation. Ill-structured democratic problems demand more than factual precision; they call for intersubjective reasoning: context-sensitive judgments t...
F. Veri, Gustavo Kreia Umbelino· Proceedings of the National...· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT scientists developed a training method to generate models that simulate the effects of human activities on climate much more quickly than other approaches.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.