Explanation type and appropriate reliance in AI-assisted patent evaluation: an exploratory study across user domain familiarity.

Abstract

Explainable AI (XAI) aims to foster appropriate reliance, but its effectiveness may depend on users' domain familiarity. We conducted an exploratory study of a simplified patent-evaluation task representative of consequential decision-support settings, comparing a feature-based (SHAP) and an example-based (two-case contrastive) explanation display with 9 domain-familiar and 24 lay participants. In both groups, the example-based condition showed a significant within-condition improvement of final over initial decisions on the same items, with no detected difference in gains between conditions. Among domain-familiar participants, switching to correct AI advice given disagreement was substantially lower under the feature-based than the example-based display, despite their stated preference for the feature-based format; among lay participants, no differences in switching to correct AI advice were detected once disagreement opportunities were considered. These findings caution against preference-driven XAI design: the display domain-familiar users preferred was the one under which they most often discarded correct AI advice.