It is argued that calibration-aligned design (rather than trust maximization alone) should guide the development and assessment of high-stakes AI decision support, because reductions in reported trust do not consistently translate into commensurate changes in reliance behavior.
Abstract
AI-mediated decision support systems are increasingly deployed in domains characterized by risk, uncertainty, and time pressure. In such environments, appropriate reliance on AI recommendations requires not only initial trust formation but also dynamic recalibration when system performance fluctuates or conflicts with other information sources. Although determinants of perceived trust (e.g., explainability, authority cues, and ethical framing) have been widely studied, less attention has been given to how reliance behavior adjusts following observed system error. This paper presents a focused qualitative synthesis of empirical studies examining trust and reliance in AI-based decision support under conditions of risk or informational divergence. Across the included studies, trust was frequently operationalized as an attitudinal construct or predictor of adoption. In contrast, fewer investigations directly measured behavioral reliance following performance degradation or assessed calibration accuracy, defined as the alignment between perceived system capability and actual performance over time. Findings suggest that reductions in reported trust do not consistently translate into commensurate changes in reliance behavior. This divergence highlights the need to distinguish attitudinal trust from behavioral calibration when evaluating AI systems in safety-relevant contexts. We argue that calibration-aligned design (rather than trust maximization alone) should guide the development and assessment of high-stakes AI decision support.
While participants rated hedged and unhedged AI as equally trustworthy and likely to be correct, they were significantly less likely to follow hedged advice in a binary choice, and how linguistic markers can be used to calibrate user reliance to model certainty is discussed.
Laura Spillner, Johanna Rockstroh, Nina Wenig et al.· International Conference on...· 0 citations
The rapid adoption of artificial intelligence (AI) in labor-intensive manufacturing raises concerns about how trust between humans and AI develops under production pressure. This study examines the erosion and consequences of human–AI trust in garment factories, where workers must quickly adapt to AI-driven systems in highly monitored environments. Drawing on the Swift Trust Theory and the Job Demands–Resources model, we propose a framework that considers relationships among constructs, such as compressed trust formation, trust fragility, sacrificial compliance, perceived organizational support, and workplace techno-pressure. We employed a two-phase mixed-methods design. An exploratory qualitative study informed construct development, followed by a quantitative study for scale validation and hypothesis testing. Results show that compressed trust formation is positively associated with trust fragility, and both are positively linked to sacrificial compliance. Trust fragility partially mediates the relationship between compressed trust formation and sacrificial compliance. Perceived organizational support weakens the relationship between compressed trust formation and trust fragility, whereas workplace techno-pressure strengthens the relationship between trust fragility and sacrificial compliance. The findings suggest that trust formed rapidly under techno-pressure can enable short-term coordination but remains structurally fragile and may convert into self-sacrificial work behaviors. The study extends Swift Trust Theory to human–AI collaboration and embeds trust dynamics within the Job Demands–Resources model, highlighting how organizational support and techno-pressure management shape whether digital transformation supports sustainable or harmful forms of adaptation.
Surajit Bag, Muhammad Sabbir Rahman, S. Alam· IEEE transactions on enginee...· 0 citations
With the growing use of artificial intelligence (AI) in public governance, understanding public willingness to delegate decision-making authority to algorithmic systems has become a key issue. While prior research has examined the relationship between trust in public institutions and trust in AI, the role of institutional trust in shaping willingness to delegate high-stakes decisions to AI remains understudied. This study aims to address this gap using nationally representative survey data from Wave 152 of the Pew Research Center’s American Trends Panel (August 2024, n = 5,410).
The study uses weighted logistic regression to assess whether confidence in the US federal government’s ability to effectively regulate AI predicts citizens’ willingness to entrust AI with important decision-making responsibilities. The analysis is based on 2,940 valid responses after excluding non-substantive answers.
The findings demonstrate that institutional trust is a statistically significant predictor of support for algorithmic delegation. Higher levels of confidence in governmental AI regulation were associated with substantially higher odds of supporting the delegation of important decisions to AI systems (OR = 1.33; 95% CI [1.19, 1.50]; p < 0.001). Although utilitarian evaluations of personal benefit exert the strongest influence, institutional trust remains significant even after controlling for sociodemographic, informational, affective factors and political predispositions.
The cross-sectional design and reliance on self-reported measures limit causal inference. The dependent variable captures normative willingness to delegate rather than the observed behavior, which is appropriate given that institutional-level AI use in higher domains is still emerging. Nevertheless, the use of national survey weights and extensive controls enhances the robustness of the findings. The results contribute to the literature on digital governance by identifying institutional trust as an independent legitimacy mechanism in the acceptance of algorithmic authority.
For policymakers, the findings suggest that public support for AI-driven governance depends not only on the performance or perceived benefits of AI systems but also on citizens’ confidence in governmental regulatory capacity. Given that AI awareness was independently associated with higher support for delegation (OR = 1.36), strengthening institutional transparency, regulatory credibility and public AI literacy may be essential for sustainable AI implementation.
As governments increasingly rely on algorithmic systems in high-stakes domains, the findings suggest that institutional trust may be an important condition for the democratic legitimacy and public acceptance of digital transformations.
This study advances research on AI governance by empirically demonstrating that institutional trust in regulatory competence functions as an independent political condition for delegating authority to algorithmic systems. Unlike prior work that examines institutional trust as one predictor among many or that measures cross-national trust differences without testing the delegation pathway, this paper theorizes institutional regulatory trust as the central legitimacy mechanism and uses normative willingness to delegate, rather than abstract approval, as the outcome.
Akniyet Nugmanova, B. Gabdulina· Transforming Government: Peo...· 0 citations
Introduction Artificial intelligence (AI) is increasingly used in public agencies to route inquiries, screen eligibility, support caseworkers, and automate routine service encounters. Citizen acceptance of these services depends on their links to public authority, accountability, and visible opportunities for human recourse. This study examines a trust-based mechanism connecting institutional trust, risk perception, AI service trust, and behavioral intention in China's digital government context. Methods The study combined an LLM-driven agent simulation involving 936 agents across three independent seeds, a 3 × 3 factorial scenario experiment involving 900 simulated agents, and a human-validation pilot using the same questionnaire and scenario structure. The pilot generated 189 submitted records, of which 182 were retained after attention checking. Results In the synthetic calibration, institutional trust is positively associated with AI service trust (IT → AST β = 0.607) and negatively associated with risk perception (IT → RP β = −0.271); risk perception is negatively associated with AI service trust (RP → AST β = −0.459); and AI service trust is positively associated with behavioral intention (AST → BI β = 0.424). The same directional pattern appears in the human-validation pilot (IT → AST β = 0.357; IT → RP β = −0.240; RP → AST β = −0.513; AST → BI β = 0.650). Scenario means also align with the simulation pattern (Pearson r = 0.803 for AST and r = 0.875 for BI across the nine cells), with the lowest pilot AST (3.667) and BI (3.413) in the fully automated high-risk condition. Discussion The findings connect confidence in government institutions with service-specific trust and indicate that perceived risk constrains acceptance of AI-enabled public services. In high-stakes automated settings, visible arrangements for human review may be necessary for AI service trust to translate into intended use. Public-sector AI acceptance is therefore shaped jointly by institutional credibility, perceived risk, and service encounter design.
Huihui Wang, Shixin Zhu· Frontiers in Psychology· 0 citations
Generative AI is increasingly used in work settings, where users often iteratively refine prompts to obtain outputs that match their intentions, potentially increasing cognitive workload. Although trust in AI is considered important for effective human--AI collaboration, how trust relates to cognitive workload---and which trust components matter most---remains unclear. This study experimentally examined trust--workload relationships in prompt-based interaction with an image-generation system. Twenty-three employees performed task-oriented image-generation tasks under two interaction conditions (Automatic vs. Prompt) designed to induce workload differences. Trust was measured using eight MDMT Performance Trust items, and cognitive workload was assessed using the Gas Tank Questionnaire. Analyses proceeded in three steps: (1) item-level correlations, (2) structural equation modeling (SEM) of Performance Trust predicting cognitive workload while controlling for Condition and Theme, and (3) a trust-items-only regression reporting standardized coefficients (\(\beta\)) with 95\% confidence intervals. SEM showed that higher Performance Trust was associated with lower cognitive workload (\(\beta=-0.385, p<.001\)), explaining 35.0\% of the variance (\(R^2=0.350\)). Item-level regression further indicated unequal contributions among trust components. These findings suggest that strengthening Performance Trust and prioritizing workload-relevant trust components can support low-burden human--AI collaboration.
Mari Saito, Seiji Yamada· AHFE International· 0 citations
ObjectiveThis study examined whether cognitive load produces selective effects on different trust updating pathways in AI-assisted decision making.BackgroundAlthough cognitive load affects trust in automation, its influence on the mechanisms of trial-by-trial trust updating remains unclear.MethodsA dual-task paradigm embedded in a mining exploration task manipulated cognitive load while capturing dynamic trust calibration. Guided by a dual-pathway framework, we operationalized process-based (analytical evaluation of AI recommendation correctness) and outcome-based (heuristic reliance on task outcomes) trust updating pathways. Trust dynamics and behavioral reliance were examined using linear mixed-effects models.ResultsCognitive load shifted the relative influence of the two trust updating pathways. Process-based updating was attenuated under high cognitive load, indicating reduced sensitivity to AI recommendation correctness during trust updating. Outcome-based information gained greater influence under high load, amplifying outcome-driven bias regardless of recommendation correctness. Asymmetric trust updating was evident overall, although the influence of cognitive load on this asymmetry depended on task outcomes. Overall, high cognitive load elevated both subjective trust and behavioral reliance on AI.ConclusionCognitive load shapes trust calibration through mechanism-level reconfiguration rather than global impairment. By revealing how cognitive constraints rebalance dual trust pathways-weakening analytic evaluation while amplifying heuristic outcome reliance-this study advances theoretical understanding of dynamic trust in human-AI collaboration.ApplicationThe results provide practical guidance for the design of AI systems in high-stakes settings, highlighting the need to support analytic trust updating and mitigate over-reliance under cognitive strain.
Xiaojiao Chen, Yonghan Liu, Yiran Ma et al.· Human Factors· 0 citations
