Skip to content
Preprint

SCP-NL2TL: Selective Conformal Prediction with Semantic Verification for Natural Language to Temporal Logic Specifications

Aug 2026 · 1 citation · 61 references
Computer Science

TL;DR

This work establishes a foundation for trustworthy natural language interfaces by enabling AI systems to recognize when generated specifications may not be reliable, and demonstrates improved translation reliability, robustness under the evaluated cross-tier shifts, and effective uncertainty-aware abstention.

Abstract

Translating natural language instructions into machine-interpretable formal specifications enables robots and autonomous systems to plan, reason, and formally verify their behavior. However, existing translation models typically generate a specification for every input, even when the result is unreliable or fails to capture the user's intent, creating risks in safety-critical applications. Inspired by selective conformal prediction, we propose a selective translation framework that not only generates formal specifications but also determines when they can be trusted. Reliability is scored by two complementary black-box signals, the fidelity of the specification back-translated into natural language and the dispersion of repeated translations under exact semantic equivalence, which fail on different errors and jointly separate incorrect translations more sharply than either alone. Conformal risk control calibrates this score into a decision that accepts a specification or abstains, with a distribution-free bound on the rate at which incorrect specifications are accepted for execution, and a conformal anomaly detector on instruction embeddings screens out-of-distribution inputs before any translation is attempted. The proposed framework is general across formal specification languages, with experiments on Signal Temporal Logic (STL), Linear Temporal Logic (LTL), and geometric Spatio-Temporal Logic (SpaTiaL) demonstrating improved translation reliability, robustness under the evaluated cross-tier shifts, and effective uncertainty-aware abstention. This work establishes a foundation for trustworthy natural language interfaces by enabling AI systems to recognize when generated specifications may not be reliable.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Feedback Makes Perfect: A Closed-Loop Framework for NL-to-STL Translation

Signal Temporal Logic (STL) enables rigorous verification and control of cyber-physical systems, but writing correct specifications requires expertise that most requirement holders lack. Large language models can translate natural-language (NL) requirements into STL, yet stronger translators alone approach an accuracy...

Bo-Wen Ye, Xiang Yin · 0 citations
Book Open access Oct 2026

From Natural Language to Formal Specifications: A Metamodel-Driven LLM Pipeline

This case study assesses the feasibility of combining LLM-based extraction, metamodel-guided refinement, automated validation, and human correction for requirements formalization and evaluates the approach on the ten published ACAS Xu safety properties.

Sally Ismail, Patrick Tessier · 0 citations
#machine learning Preprint Sep 2026

RAISE: Reinforcing Access Control Policy Synthesis in LLMs via Symbolic Evaluation

RAISE is introduced, which trains policy synthesizers from formal verification in two stages, verified supervised fine-tuning (SFT) followed by a reinforcement learning (RL) stage that learns from verifier signal, and it is found that SFT succeeds largely by letting models express authorization logic they already have.

Ying-Ming Zhou, Adarsh Vatsa, William Eiers · 0 citations
#artificial intelligence Preprint Aug 2026

CEDAR: Automata as Verifiable Interfaces for Language-Guided Embodied Action

CEDAR is presented, a counterexample-guided framework that grounds instructions as regular languages over environment event traces and represents both skills and specifications as deterministic finite automata, suggesting that regular languages offer a practical verification layer between natural-language instructions...

Le Chen, Alvaro Velasquez, Ashutosh Trivedi · 0 citations
#small language model Book Open access Oct 2026

Formally-Verified Training and Runtime Validation for Small Language Models Using Relational Specifications

This work investigates whether relational specifications can ground SLM behaviour by mathematical proof rather than statistical regularities alone, and gives practitioners a route to deploy verifiable SLMs in safety-critical settings.

Chia Sabah · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.