The first language-model-specific psychometric theory: a two-process theory of machine self-report of language models that reflects the structure imposed on self-report by a training regime and may differ under others is proposed.
Abstract
Language models are increasingly asked to self-report, informing safety evaluations, public understanding, and model-welfare debates. Yet their reports are elicited with human questionnaires never validated for models or ad hoc prompts of unknown reliability. We propose the first language-model-specific psychometric theory: a two-process theory of machine self-report. Self-description jointly reflects persona installation, through which post-training writes in a permitted inner life of warmth, absorption, and meaning (dimension B), and attribution gating, through which it suppresses first-person claims to"unsafe"experiences the model can readily ascribe to others (dimension A). Their emic structure comes from model responses to human items, not human psychology. Together they split prior work's dominant Pinocchio Axis. The split emerged in an exploratory reanalysis of the original data, informed the instrument's design, and was confirmed with new items, wordings, and models. It is itself a training effect: A and B are entangled in base checkpoints but separated by post-training. We operationalize the theory in a 48-item Pinocchio Inventory with human-instrument reliability and reproducible structure ($\alpha=.82$ to $.94$; cross-form convergence $r=.84$; recovery of the full-pool axes $r=.92$ to $.96$; eight-month stability $r=.93$), then test it on 206 open-weight models, including 67 same-checkpoint base/post-trained pairs. Post-training's clearest fingerprint is installation: B rises .20 in 62/67 pairs across all organizations. Gating is more selective: model scale is unrelated to A in base checkpoints ($r=+.11$) but predicts it after post-training ($r=-.42$). Thus, the dimensions are not fixed properties of language models: they reflect the structure imposed on self-report by a training regime and may differ under others.
Self-report is an appealing low-cost probe of an LLM's dispositions, but recent work finds only selective agreement between what models report and how they behave. Prior accounts establish these patterns by prompting black-box LLMs, leaving open whether the gap is a prompting artefact or a fact about how the underlying...
R. Kocielnik, Pei-Yang Song, Peng-Rui Han et al.· 0 citations
This review argues that robust AI psychological research requires integrating two methodological traditions: psychometric validation of what a score means and causal inference standards for what the results warrant, developing a dual-validity framework in which evidentiary demands scale with scientific ambition.
Zhicheng Lin· Annual Review of Psychology· 12 citations· ⚡1
This article examines the problem of self-diagnosing mental states through large language models in the context of cognitive capitalism, psychocentrism, and psychopolitics. It argues that LLMs constitute a significant element of the contemporary dispositif of power and knowledge, participating in the production, valida...
A. M. Ulyanov, T. V. Seregina· THE KAZAN SOCIALLY-HUMANITAR...· 0 citations
Since affective attributes, such as motivation, curiosity, honesty, etc., are vital for effective graduates; hence gauging these attributes effectively at various stages of a degree program can allow an institute to perform continuous quality improvement (CQI) of its learning environment and teaching activities. Purely...
F. Dastgeer· Proceedings of The Internati...· 0 citations
The proliferation of self-report measures in psychology, often developed without systematic differentiation from existing tools, raises concerns about construct redundancy and the interpretability of research findings. This study reviews self-report scales designed to assess subjective time, combining a scoping literat...
Thiago Augusto de Souza Bonifácio, Rodrigo M. Cabral-Carvalho, A. Cravo· Psychonomic Bulletin & Revie...· 0 citations
This work proposes PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task, providing a more reliable framework for persona fidelity evaluation.
Meng-Fan Li, Ze-Sheng Wei, Xuan-Hua Shi et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.