A coevolutionary account of normative human–AI interaction
TL;DR
It is argued that the integration of aligned large language models into social practice reshapes the structure of moral agency itself, embedding derivative normative orientations that recursively feed back into human moral formation.
Abstract
This paper develops a coevolutionary account of extended morality in AI-mediated societies. Drawing on 4E cognition and theories of human–AI coevolution, it argues that the integration of aligned large language models into social practice reshapes the structure of moral agency itself. Alignment functions as technological habituation, embedding derivative normative orientations that recursively feed back into human moral formation. The paper specifies constitutive conditions—reliable availability, default endorsement, functional integration, and counterfactual impairment—under which such coupling counts as genuinely extended morality rather than mere assistance or scaffolding, and distinguishes normative authority, which remains asymmetrically human, from normative mediation and stabilization, which become sociotechnically distributed. Engaging mediation theory, distributed morality, and empirical research on AI-induced behavioral change, it addresses objections from simulation and moral deskilling. It concludes by outlining governance conditions and the reflexive virtue of AI maturity required for responsible moral coevolution.