Mediational Opacity: How AI Reshapes Our Epistemic Environments, and How to Audit It

Abstract

AI systems do more than operate within our epistemic environments. They actively reshape them by modifying what users treat as relevant evidence, how they come to know things, and even the standards by which they judge whether a system can be trusted. We argue that this reshaping is largely hidden from view, and we call the resulting condition 'mediational opacity'. It arises when an AI system whose inner behaviour resists inspection stops being a tool we consciously pick up and use, and instead becomes an environment we inhabit. This type of opacity is different from the two problems most current work addresses, namely the 'black-box problem' that explainable AI tries to 'open up' and the question of whether individual outputs are fair or accurate, which responsible-AI frameworks track. It is something prior to both, and related precisely to the fact that a deployed system quietly restructures the way its users think and what they count as good judgement. Four reinforcing features (phenomenological silence, dynamic self-reinforcement, societal scale, and contextual promiscuity) make this condition especially hard to detect. Because making AI systems more explainable is not enough to address these features, we propose mediation audits as a complementary approach, focused on the relationship between system, user, and environment, structured along five dimensions, and grounded in a capability approach extended to treat AI as an environmental factor. Finally, we show that agentic AI radicalises the problem since by delegating the 'thinking' itself, it produces chains of intermediate epistemic operations that no human can directly observe.