Stuck on Suggestions: Automation Bias, the Anchoring Effect, and the Factors That Shape Them in Computational Pathology
TL;DR
The findings indicate that AI integration generally enhances diagnostic performance, but also introduced a 7% automation bias rate, quantified as the number of accepted negative consultations, where a previously correct independent assessment gets overturned by inaccurate AI guidance.
Abstract
Artificial intelligence (AI)-driven clinical decision support systems (CDSS) hold promise to improve diagnostic accuracy and efficiency in computational pathology. However, collaboration between human experts and AI may give rise to cognitive biases, such as automation and anchoring bias, wherein users may be inclined to blindly adopt system recommendations or be disproportionately influenced by the presence of AI predictions, even when they are inaccurate. These biases may be exacerbated under time pressure, pervasive in routine pathology diagnostics, or shaped by individual user characteristics. To investigate these effects, we conducted a web-based experiment in which trained pathology experts (n = 28) estimated tumor cell percentages twice: once independently and once with the aid of an AI. A subset of the estimates in each condition was performed under time constraints. Our findings indicate that AI integration generally enhances diagnostic performance. However, it also introduced a 7% automation bias rate, quantified as the number of accepted negative consultations, where a previously correct independent assessment gets overturned by inaccurate AI guidance. While time pressure did not increase the frequency of automation bias occurrence, it appeared to intensify its severity, as evidenced by a performance decline linked to increased automation reliance under cognitive load. A linear mixed-effects model (LMM) analysis, simulating weighted averaging, revealed a statistically significant positive coefficient for AI advice, indicating a moderate degree of anchoring on system output. This effect was further intensified under time pressure, suggesting that anchoring bias may become more pronounced when cognitive resources are limited. A secondary LMM evaluation assessing automation reliance, used as a proxy for both automation and anchoring bias, demonstrated that professional experience and self-efficacy were associated with reduced dependence on system support, whereas higher confidence during AI-assisted decision-making was linked to increased automation reliance. Together, these findings underscore the dual nature of AI integration in clinical workflows, offering performance benefits while also introducing risks of cognitive bias–driven diagnostic errors. As an initial investigation focused on a single medical specialty and diagnostic task, this study aims to lay the groundwork for future research to explore these phenomena across diverse clinical contexts, ultimately supporting the establishment of appropriate reliance on automated systems and the safe, effective integration of human–AI collaboration in medical decision-making.
