Decision substitution and the reorganisation of judgement in AI-mediated society
TL;DR
A society-shaping-the-algorithm perspective is advanced that reframes human-in-the-loop governance as a collective and institutional condition rather than a procedural safeguard, providing a conceptual foundation for preserving meaningful human judgement in AI-mediated societies.
Abstract
Artificial intelligence (AI) systems are increasingly embedded in institutional and everyday contexts, where they are publicly framed as decision-support tools yet often function as de facto decision-makers. This study argues that the shift from algorithmic support to decision substitution constitutes a significant societal problem insufficiently addressed by prevailing AI ethics and governance frameworks. Rather than focussing solely on technical performance, bias, or transparency, the analysis examines how AI-mediated practices reconfigure human agency, responsibility, and judgement. Drawing on philosophy of technology and socio-technical scholarship, it conceptualises AI systems as cultural and epistemic infrastructures that shape how knowledge is produced, authority distributed, and action legitimised. It argues that AI increasingly reorganises the decision field by restructuring visibility, validation, participation, and legitimacy, thereby transforming the conditions under which judgement is exercised. The research distinguishes between decision support, decision delegation, and decision substitution, showing how human judgement can be displaced even when human actors remain formally involved. The analysis identifies threshold conditions through which decision substitution emerges, including constrained dissent, diminished corrective capacity, default algorithmic legitimacy, and the separation of authority from accountability. These dynamics generate responsibility asymmetries in which accountability remains attached to human actors while authority migrates towards algorithmic systems. The problem is most acute in contexts characterised by indeterminacy, where values, tacit knowledge, and ethical concerns resist formalisation. The research concludes by advancing a society-shaping-the-algorithm perspective that reframes human-in-the-loop governance as a collective and institutional condition rather than a procedural safeguard, providing a conceptual foundation for preserving meaningful human judgement in AI-mediated societies.