This work proposes A-B-D to infer traits bottom-up from behavioral data (B-data), namely how agents act on their environment and communicate with users, as recorded in existing trajectories, and offers a new lens for understanding AI personality.
Abstract
Users increasingly describe different AI agents as distinct colleagues to work with. AI personality research aims to quantify such impressions by attributing human- like"traits"to agents. However, existing measures fall short: models'self-reports (S-data) diverge from their actual behavior, while informant ratings from LLM judges (I-data) are costly to scale and cover few everyday scenarios. In this paper, we propose A-B-D to infer traits bottom-up from behavioral data (B-data), namely how agents act on their environment and communicate with users, as recorded in existing trajectories. From 345,667 real-world trajectories spanning 80 models, 12 tasks, and 50 harnesses, we extract 318 candidate features that capture both the actions an agent takes at each step (functional) and the language accompanying them (linguistic). We retain only features that show instance-level stability, cross-task consistency, and model discriminability. Factor analysis of the remaining 79 features uncovers six stable, model-attributable factors, two functional and four linguistic. For example, Claude-Sonnet-4.6 and Kimi-K3 exhibit the most planfulness, whereas GPT-5.5 and GPT-5.6 are the least energetic. Moreover, we quantify the"knowledge-action gap"in the wild: these factors correlate only weakly with self-reported Big Five scores, even for conceptually matched pairs such as extraversion and energetic (r = 0.07, p = 0.58). Our work offers a new lens for understanding AI personality, with implications for users, developers, and researchers from both computer science and social science.
A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its i...
Yi-Lin Geng, Omri Abend, Eduard H. Hovy et al.· Conference on Empirical Meth...· 0 citations
Large language model (LLM) agents assist users with everyday tasks that can be completed in many reasonable ways. Even when their answers are useful, how agents carry out these tasks may not match users'preferences and needs. For example, agents differ in whether they ask clarifying questions or search the web. We intr...
Woojung Song, Hoyeol Yang, Jeonghoon Shim et al.· 0 citations
Current approaches to evaluating personality in large language models (LLMs) typically prompt them to self-report on psychological questionnaires such as the Big Five Inventory. However, these methods assess introspective labels rather than observable behavior, despite the fact that LLMs are deployed to act in realisti...
Tao Pu, He-Xi Wang, Ze-Yang Liu et al.· Proceedings of the Thirty-Fi...· 0 citations
Different capacities for mentalization across LLMs are demonstrated, and cognitive computational modeling is highlighted as a formal method for assessing comparative intelligence across humans and machines.
Aamir Sohail, Xintong Zhong, Arkady Konovalov et al.· Research Square· 0 citations
An automated, data-driven approach to uncover patterns, which the authors term traits, of effective human-AI interaction that are aligned with task outcomes is explored and Principal Trait Analysis is proposed, a Principal Component Analysis-inspired algorithm for deriving common traits from patterns in LLM conversatio...
Hunter McNichols, Kai Du, Andrew S. Lan· 0 citations
Large language models (LLMs) are increasingly used as synthetic personas representing survey respondents. Their validity as substitutes for particular respondents depends on whether they reproduce individuals'decisions. We examine what information helps synthetic respondents predict each individual's later choices, usi...