RightsGen, a framework that leverages Large Language Models (LLMs) to automate the discovery and documentation of user rights declarations in Android applications, demonstrates high efficiency in dynamic testing and highlights the effectiveness of RightsGen in automating the generation of user rights declarations.
Abstract
Ensuring compliance with privacy regulations, such as GDPR and CCPA, has become a critical challenge for mobile applications. These regulations mandate that apps provide accessible paths for users to exercise rights, such as account deletion and data export. However, these paths are often buried deep within complex UI hierarchies, making manual auditing unscalable. In this paper, we introduce RightsGen, a framework that leverages Large Language Models (LLMs) to automate the discovery and documentation of user rights declarations in Android applications. Unlike traditional random UI fuzzers, RightsGen utilizes the semantic reasoning capabilities of LLMs to navigate dynamic UI states. It employs a path-oriented mechanism and a GUI Agent acting as a rights tester guided by structured rights tuples. In our evaluation across 144 popular apps, RightsGen demonstrated high efficiency in dynamic testing, achieving a recall rate of 92.5% in recognizing user rights paths and 87.3% in extracting rights tuples. These results highlight the effectiveness of RightsGen in automating the generation of user rights declarations.
The advent of LLMs has significantly changed the research on privacy policy and data compliance analysis by enabling tasks that previously required specialized, domain-specific tools. However, it remains unclear to what extent LLMs can truly replicate the diverse functionalities, and the wide range of methodologies and analysis offered by prior work. In this paper, we conduct the first systematic evaluation of whether off-the-shelf LLMs can replace specialized privacy analysis tools. We study six representative tools spanning three major functionalities: contradiction detection, regulatory compliance analysis, and privacy policy summarization and aggregation, and across three intermediate tasks: structured data extraction using tuples, Semantic Role Labeling (SRL) and manual privacy policy labeling. We compare the performance of two state-of-the-art LLMs (GPT-5.2 and Gemini-2.5 in various configurations) against the tools by directly prompting the models to perform corresponding functionalities and tasks on a custom dataset of 10 privacy policies, allowing us to assess whether off-the-shelf models can produce tool-specific functionalities without further engineering or domain-specific training, major limitations in prior work. Our results show that LLMs consistently match or exceed the capabilities of existing tools across the functionalities. In manual labeling of first-party collection entities, LLMs achieved an average precision of 81.8% and recall of 70.9%, while for labeling of third-party sharing entities, they achieved an average precision of 91.4% and recall of 70.8% compared to the OPP-115 dataset. Overall, our findings indicate that LLMs can effectively perform a broad range of functionalities and tasks in privacy policy and regulation analysis that previously required specialized tools.
Madhav Aryal, Sudipa Saha, Sunil Manandhar et al.· 0 citations
This work presents the first privacy- and security-oriented measurement of M365 third-party applications, and finds that only 1,069 of them expose both descriptions and permission sets, with significant inconsistencies in transparency across official distribution channels.
Vincenzo Longo, Alberto Verna, Nikhil Jha et al.· 0 citations
Whether off-the-shelf Large Language Models (LLMs) can effectively reason about taint flows in Android apps is investigated, and preliminary findings suggest that LLM reasoning may effectively complement traditional static taint analysis.
Nicholas Miazzo, Marco Alecci, Jordan Samhi et al.· 0 citations
Large Language Models (LLMs) are now widely used for code generation, yet even syntactically correct output may contain logical and semantic errors that remain invisible until runtime, particularly in framework-driven applications, where correctness depends on dependency injection, framework conventions, configuration, library compatibility, and database interaction. This paper presents GenTest, an open-source platform for context-aware dynamic generation of Java code using LLMs, runtime compilation and class loading, Spring bean registration, JPA/PostgreSQL execution, and declarative YAML-based test validation. GenTest supports multiple LLM providers through a provider-agnostic interface and evaluates generated components within a live Spring Boot application context. Across 209 test cases and 933 assertions, GenTest achieves a 63.9% assertion pass rate, and 94.2% of failures occur after successful compilation, confirming that execution-centered evaluation reveals failure modes invisible to static analysis.
Muhammed Suphi Şeyhkuruş, Tolga Ovatman· Annual International Compute...· 0 citations
This paper presents OS-NLI, a privacy-respecting and modular natural-language interface for operating-system tasks. Unlike vendor-bound assistants, OS-NLI separates intent interpretation from command execution through an architecture comprising a web interface, a retrieval-augmented generation (RAG) layer, an API, an encrypted datastore, and a local execution worker. The LLM is treated as an untrusted suggestion engine: only pre-approved command templates with validated parameters may be queued, and execution requires explicit user confirmation. Sensitive data are protected with TLS in transit and AES-GCM at rest, while the threat model explicitly excludes hosts that are fully compromised at runtime. A prototype was implemented across Windows 11 and Fedora Linux. Evaluation on 60 natural-language queries shows strong task performance for application and utility identification, median answer latency of 1.4 s, and median post-confirmation command-execution latency of 285 ms. Preliminary malicious-input checks indicate that non-allowlisted outputs are rejected before execution. The current prototype still relies on vendor-hosted components for the UI host and LLM, so it is not yet end-to-end auditable; however, the architecture isolates these dependencies and provides a practical migration path to self-hosted deployment. These results support the feasibility of a safer, auditable design for naturallanguage OS interaction.
David Lawrence, Xiaoli Huan· International Conference on...· 0 citations
Microservice architectures are inherently plagued by "authorization blindspots"–divergent security policies across independent services that create undetectable downstream security drifts. As systems evolve, these invisible vulnerabilities leave applications highly susceptible to privilege escalation and catastrophic data breaches. To eliminate these blindspots, we introduce a novel, fully automated framework that bridges the precision of formal static analysis with the adaptiveness of Generative AI. By extracting a policy-enriched Intermediate Representation of the microservice system, our approach deterministically guides GPT-5 to synthesize executable, downstream-aware policy test suites targeting specific policy inconsistencies. Evaluation on the Train-Ticket benchmark denotes that our method outperforms state-of-the-art tools such as EvoMaster and EvoSuite by generating 100% semantically valid authorization policy tests. Further, this research provides vital empirical validation for formal methods. By producing 97.4% error-free drift validation tests, our approach systematically neutralizes static analysis noise. Ultimately, these results establish a rigorous, highly effective pathway for hybridizing formal structures with Large Language Models to definitively verify complex, distributed authorization policies.
Md Arfan Uddin, Shakthi Weerasinghe, Connor Wojtak et al.· International Symposium on S...· 0 citations