When Safety Claims Fail Silently: A Modelling Approach to AI Assurance

Abstract

Problem. The rapid deployment of large language models (LLMs) into safety-critical roles, including mental health support, education, and professional guidance, has created a gap between the safety claims AI developers publish and the behaviors their systems exhibit in practice. Current documentation practices, primarily model cards and system cards, describe safety properties in natural language which is unstructured by nature, lack proper validation mechanisms, and are rarely maintained across the lifecycle of a deployed system. Proposal & Approach. This paper presents my doctoral research project that addresses this gap through a modelling language and semantics to present AI safety claims as dynamic, reusable models. It will be developed through two phases. First, we propose to model AI safety claims with a structured argumentation notation that makes the logical dependencies between evidence and claims explicit and exposes precisely where a safety argument breaks down. Second, we propose a guidance system built on top of JPipe, an open-source modelling tooling platform, that allows practitioners to query a curated library of modeled safety claims and receive contextually relevant, personalized guidance for their own AI systems. Current Status. The first phase is substantially complete: five publications establish the empirical and tooling foundations of the project, including the extraction and categorization of 32 recurring safety patterns from major provider model cards, an in-depth model-based analysis of the GPT-5 system card, and the development of composition operators within JPipe that enable decomposition and reuse of sub-models. The remaining work focuses on completing the models library, building the guidance system, and evaluating both phases through expert review, case studies, and a user study with ML practitioners and safety-critical systems engineers.