Democratizing AI Responsibly Through Research Software: A Case Study in Suicide Prevention Research

Abstract

The rapid democratization of AI and machine learning (ML) capabilities–the broadening of access beyond specialists through reduced computational and resource barriers–is reshaping scientific research [1]. In particular, low-code and no-code research software that allows users to incorporate AI/ML into their workflows have enabled a broader community of nontechnical researchers to leverage advanced computational techniques across disciplines. However, expanded access to AI/ML capabilities does not inherently democratize the expertise required to responsibly evaluate outputs in ways that mitigate potential risks [2], especially given that many existing frameworks for ethical AI/ML principles are high-level and provide little operationalized guidance [3]. Suicide prevention is one such field in which expanded access to AI/ML methodologies has the potential to benefit scientific research, but responsible deployment is especially critical. ML methods have shown promise in revealing the complex risk and protective factors underlying suicide, often outperforming traditional statistical approaches [4], as well as informing culturally and demographically responsive interventions [5]. At the same time, the risks associated with broadened access are especially acute in high-stakes domains such as this, where research results may influence clinical decision-making, public policy, and resource allocation. We present the Suicide Risk Dashboard (SRD) as a case study in demonstrating the software requirements and design patterns that research software engineers (RSEs) could leverage for the responsible democratization of AI in science to mitigate risks in sensitive domains. The SRD is a no-code data analysis platform that supports the exploration and analysis of large-scale, suicide-related public health data. To enable researchers with varying levels of computational expertise to audit analysis results, the SRD provides a guided interface for constructing and training custom predictive models that follows responsible-AI-by-design principles [3], embedding as product features explainability techniques, such as global and local explanations using SHAP; fairness assessments, including subgroup performance comparisons and equalized odds tests; interpretable reporting based on user-centered design practices; and accessible visualizations. As AI capabilities become increasingly democratized, RSEs face the challenge of abstracting complex computational methods through usable research software without obscuring data and analysis assumptions, limitations, and evaluation practices needed for responsible use by end users. This presentation provides examples and design considerations to address this challenge that may be adapted across scientific domains, and demonstrates that research software can help disseminate not only AI/ML capabilities, but also the methods and practices required to translate those capabilities into trustworthy scientific impact.