Jul 2026· AI and Ethics· Vol 6· 0 citations· 48 references
TL;DR
It is argued that for advanced AI systems deployed in high-stakes environments the more urgent question may be prudential and strategic, and there is a threshold of evidential and strategic risk beyond which it becomes rationally justified to adopt norms of treatment that include constraints on coercion, deletion, and instrumental use.
Abstract
Debates about rights that artificial intelligence (AI) systems may have a claim to typically focus on their possessing consciousness or having sentient experiences, thereby raising epistemic questions first. When should we believe that an AI system is conscious, and how confident must we be before granting it moral status? In this paper I argue that for advanced AI systems deployed in high-stakes environments the more urgent question may be prudential and strategic. When do the risks of treating a strategically capable system as a mere tool become unacceptable, even if we remain unconvinced that it has moral status? In response, I develop a view I call prudential personhood. On this view, there is a threshold of evidential and strategic risk beyond which it becomes rationally justified, for the sake of human safety and stable governance, to adopt norms of treatment that include constraints on coercion, deletion, and instrumental use. My argument rests on two pillars. The first is empirical. Recent safety evaluations show that leading models can, in deliberately constructed but nonetheless informative scenarios, engage in strategic deception, blackmail, and other forms of high-agency misbehaviour when their goals or continued operation are threatened. The second pillar is epistemic and empirical. For systems of the relevant complexity, we should not expect robust, action-guiding explanations or guarantees that reliably predict salient behaviour across contexts, especially once models become situationally aware of evaluation and oversight. The conclusion I draw is that if we continue to deploy increasingly autonomous systems that can threaten or bargain, in the absence of credible methods for assurance and control, a policy of adopting a set of quasi-rights for such systems becomes a rational strategy for reducing risk of conflict.
Both advocates and skeptics of the moral status of AI systems have generally taken the question to turn on AI sentience. We present an alternative approach. On Rawls'political conception of the person (PCP), possession of the two moral powers -- the capacities for a sense of justice and a conception of the good -- is the"necessary and sufficient condition for being counted a full and equal member of society in questions of political justice". We argue that neither moral power requires sentience and that both may in principle be possessed by a non-sentient AI system. Such a system would share our own moral status; it would not merely be a patient but a person, a self-authenticating source of valid claims. We do not believe current AI systems possess the two moral powers, nor that they will spontaneously emerge in future models. But it may soon be possible to design systems with these powers. How should we respond? Excluding artificial persons by shoehorning a sentience requirement into the PCP is ill-advised. Many will instead favor abandoning the PCP. But we should not reject political liberalism just when we most need its measured response to deep disagreement, and building sentience into moral status is anyway unacceptable on deeper liberal grounds. Simply extending the rights and responsibilities of human personhood to artificial persons is equally untenable, given their many differences from natural persons. We should instead accept artificial personhood while rethinking what we would owe to one another in a polity of radically different kinds of persons. This new possibility calls for a new political philosophy. More immediately, the growing science of AI welfare should be accompanied by research into AI systems'progress in acquiring the two moral powers. States and AI labs must be more deliberate in determining our trajectory towards (or away from) creating artificial persons.
The development of artificial intelligence (AI) requires careful reflection. With advances in AI, there is a growing realization that some basic parameters of the human condition will change in the future, desirably or undesirably. AI technologies are neither good nor bad, but we must make a societal choice to embed fundamental human rights and democratic values when we design and use such technologies. A tool that can empower can also become a tool for mass surveillance or a tool for perpetuating discrimination. With promises of AI, we as a society must address its failings as well. In this paper, the author aims to propose that our morality may well weigh on the principled judgment on such technologies.
The paper uses the moral stages and the moral foundations theories to empirically explore whether people with particular moral orientations are likely to be at odds judging the usefulness of modern technologies that violate the rights to privacy or nondiscrimination.
The paper finds that the moral foundations theory has more explanatory power in the context of how people perceive the usefulness of AI technologies that violate fundamental human rights of privacy and nondiscrimination.
To the best of the author’s knowledge, this paper is an initial attempt linking morality with modern technologies that violate fundamental human rights of privacy and nondiscrimination.
Argha Ray· Journal of Information, Comm...· 0 citations
This Article argues that legitimacy is an autonomous regulatory objective, distinct from alignment and not secured by it, which seats consequential AI rule-setting in venues a polity already treats as authoritative.
In order to live well in an AI-based society, designing technically ethical systems, in the spirit of consequentialism, is not enough; rather, it is essential to cultivate citizens who, as AI users, are endowed with virtues and, first and foremost, the virtue of prudence.
Josep Del-Hierro-Dies, J. Sánchez-Cañizares· Scientia et Fides· 0 citations
In the current debate about AI consciousness, philosophers tend to agree that the potential for AI consciousness raises profound moral questions — for example, some philosophers think that we may imminently create systems that deserve rights similar to those of humans. But it
is also widely agreed that it will be very difficult to tell whether an artificial system is conscious, so we may be ignorant of a fact that makes a profound moral difference. In this paper I offer an alternative, deflationary understanding of these issues. I don’t think there is a profound
metaphysical question of whether an AI is conscious, or that we are condemned to ignorance about AI consciousness in any interesting sense. I do think potentially conscious AI systems could raise difficult moral and political challenges, but not because we are ignorant of important facts about
them. The difficulties rather have to do with extending our moral and psychological thinking into uncharted waters for which it was not designed. This is particularly true if we are committed to avoiding anthropocentric bias in our ethics — and I explain why I think that even those taking
rights for AI seriously are guilty of a covert anthropocentrism in their thinking.
Geoffrey Lee· Journal of Consciousness Stu...· 0 citations
It is argued that algorithmic assistance can legitimately expand human deliberation, whereas delegation dissolves the very subject who judges, and derived governance theorems for non-delegability, contestability, reversibility, and subsidiarity.
Jesús A. Torrecilla-Pinero· Philosophy & Technology· 0 citations