Aug 2026· Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care· Vol 15, pp. 62 - 66· 0 citations· 19 references
TL;DR
Results identify a "symmetry of modality": qualitative interviews correlate with written text explanations, while Think-Aloud protocols better assess cognitively demanding tools like SHAP values.
Abstract
While AI-enabled medical devices (AIeMD) are redefining healthcare, safety relies on human-AI interaction as much as algorithmic performance. This systematic review of 53 studies reveals that 58.2% of research omits Explainable AI (XAI) methods, despite transparency being a regulatory and safety necessity. Results identify a "symmetry of modality": qualitative interviews correlate with written text explanations, while Think-Aloud protocols better assess cognitively demanding tools like SHAP values. Currently, AI-specific risks like automation bias, driven by algorithmic opacity, are critically under-reported. To ensure safe adoption, practitioners must move beyond "user satisfaction" to treat transparency and XAI as safety-critical requirements for trust calibration.
Artificial Intelligence-Enabled Medical Devices (AIeMD) promise to revolutionize healthcare, yet their safe adoption relies on effective Human-AI Interaction (HAAI) design and validation. Established usability engineering standards and guidances, including IEC 62366-1 and FDA frameworks, fail to address the novel sociotechnical risks of "black-box" systems, including automation bias and the misalignment of clinician mental models. This doctoral project directly addresses this gap, with the primary goal of developing a tailored human factor/usability engineering framework specifically for AIeMD. The project aims to establish robust methodologies for evaluating transparency and trust, integrating human factors and clinical performance into a multidimensional validation pipeline. Ultimately, this work will provide the evaluative tools necessary to move from subjective satisfaction to safety-critical, risk-based validation, ensuring that AI-enabled health solutions are clinically reliable, transparent, and compliant
Mariana de Oliveira· Information Hiding· 0 citations
This position paper presents a manifesto for a longitudinal, three-fold methodological pivot in health human-AI interaction, and proposes moving beyond static satisfaction metrics towards relational metrics —Longitudinal Trust Calibration, Automation Bias Drift, and Error Recovery Velocity—that track the maturity and resilience of the human-AI partnership.
Mariana de Oliveira, Célia F. Cruz, Nuno Matela· Information Hiding· 0 citations
A scoping review of XAI applications in telemedicine is presented, examining 20 studies published between 2022 and 2026 to characterise the methods used, their evaluation approaches, and gaps between technical explainability and clinical usability.
Artificial intelligence is increasingly used to support clinical decision making, yet concerns remain regarding algorithmic aversion, automation bias and the preservation of meaningful human oversight; while explainable AI aims to improve transparency, less attention has been devoted to the design of human–AI interaction protocols. This study investigates Frictional AI, an interaction paradigm that introduces cognitive friction to encourage critical engagement with AI recommendations. First, semi-structured interviews were conducted with a legal expert and a psychologist and analyzed through thematic analysis to identify legal, ethical, and cognitive requirements for AI-assisted decision support. Second, a user study involving 96 medical residents compared three interaction protocols: a conventional explainable AI-first design (XAI) and two friction-based protocols, namely a judicial protocol based on juxtaposed explanations (Judicial AI, JAI) and an adjunct protocol requiring an initial unsupported decision before AI exposure (AAI). Diagnostic accuracy and confidence, perceived usefulness, completion time, and reliance patterns were evaluated. The interviews highlighted the importance of human-centered explanations, contrastive reasoning, preservation of professional responsibility, and the role of user studies in evaluating human–AI interaction. The quantitative results showed that none of the AI-assisted conditions improved diagnostic accuracy relative to the no-support baseline. However, JAI achieved performance comparable to the baseline, outperforming XAI and AAI, and exhibited the lowest level of over-reliance. Overall findings suggest that the effectiveness of decision-support systems depends not only on model performance and explanation quality but also on interaction design. In conclusion, while preserving diagnostic performance, judicial protocols showed promise in mitigating automation bias and promoting active cognitive engagement in clinical decision support.
Samuele Pe, Laura Bergomi, G. Nicora et al.· Machine Learning and Knowled...· 0 citations
This mixed-methods evaluation suggests that a deliberately constrained, language-focused AI system can improve the accessibility of medical notes while preserving clinical accuracy and safety while extending into clinical interpretation.
Nicholas Lamb· Frontiers in Digital Health· 0 citations
It is argued that usability engineering should be a formative, safety-critical discipline integrated throughout the product lifecycle, including aftermarket deployment, and not merely a compliance task.
Preetha Moorthy, T. Nagel, Eva Hornecker· i-com· 0 citations
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.