A privacy-preserving hybrid middleware architecture that enforces a local trust boundary as its primary design constraint that is model-agnostic, requires no retraining of the underlying LLM, and is compatible with black-box API deployments is proposed and evaluated.
Abstract
The deployment of large language models (LLMs) in real-world applications introduces a compounding security problem: detecting adversarial inputs such as prompt injection and jailbreak-driven data leakage while simultaneously preventing the detection mechanism itself from becoming a source of data exposure. Existing approaches address either detection effectiveness or privacy preservation, but rarely both in a unified, deployable architecture. This paper proposes and evaluates a privacy-preserving hybrid middleware architecture that enforces a local trust boundary as its primary design constraint. The architecture combines deterministic rule-based screening, a fine-tuned small language model (SLM) operating entirely within the local processing environment, and a sensitivity-aware routing mechanism that invokes external LLM reasoning only for prompts all local components have assessed as non-sensitive. Evaluation on a 120-prompt benchmark spanning benign, jailbreak, and sensitive categories (including 20 hard negatives constructed to be lexically adjacent to genuine secrets) shows that the routed architecture attains 95.83% accuracy with complete recall, retaining 95% of sensitive prompts within the local boundary, at the cost of a 12.5% false-positive rate. Comparison against two published detectors reveals a systematic asymmetry: an injection-specific classifier reaches 82.5% recall on jailbreak prompts but 25% on sensitive ones, while a content-safety model inverts that profile, confirming empirically that the two risks are addressed separately by current tooling. The framework is model-agnostic, requires no retraining of the underlying LLM, and is compatible with black-box API deployments. The evaluation dataset and fine-tuned model are released publicly.
While Large Language Models (LLMs) have become essential productivity tools, their integration into workflows without adequate safeguards creates significant risks. This paper proposes an open-source, privacy-focused, user-facing firewall designed to secure both web-based and programmatic LLM interactions. The architecture combines a browser extension and a proxy for total traffic interception across both HTTP(S) and WebSocket communications. At its core, a flexible multi-agent pipeline delivers data leakage prevention through a hybrid approach combining deterministic detectors with LLM-driven semantic analysis, proprietary code leakage prevention, and extensible components designed for future security enhancements such as prompt injection evasion. The framework's layered architecture enables deployment across heterogeneous environments, allowing organizations to balance computational cost, detection depth and latency. Evaluation results demonstrate it achieves F1 scores of up to 94.93% on optimal configurations.
Hugo Garc'ia Cuesta, Pablo Mateo Torrej'on, Alfonso Sánchez-Macián· 0 citations
This paper presents OS-NLI, a privacy-respecting and modular natural-language interface for operating-system tasks. Unlike vendor-bound assistants, OS-NLI separates intent interpretation from command execution through an architecture comprising a web interface, a retrieval-augmented generation (RAG) layer, an API, an encrypted datastore, and a local execution worker. The LLM is treated as an untrusted suggestion engine: only pre-approved command templates with validated parameters may be queued, and execution requires explicit user confirmation. Sensitive data are protected with TLS in transit and AES-GCM at rest, while the threat model explicitly excludes hosts that are fully compromised at runtime. A prototype was implemented across Windows 11 and Fedora Linux. Evaluation on 60 natural-language queries shows strong task performance for application and utility identification, median answer latency of 1.4 s, and median post-confirmation command-execution latency of 285 ms. Preliminary malicious-input checks indicate that non-allowlisted outputs are rejected before execution. The current prototype still relies on vendor-hosted components for the UI host and LLM, so it is not yet end-to-end auditable; however, the architecture isolates these dependencies and provides a practical migration path to self-hosted deployment. These results support the feasibility of a safer, auditable design for naturallanguage OS interaction.
David Lawrence, Xiaoli Huan· International Conference on...· 0 citations
Large Language Models (LLMs) such as ChatGPT, Gemini, Claude, etc. have become integral tools for technical writing, software development, and communication. However, these systems, despite privacy assurances, still expose a significant privacy risk. User prompts often contain personally identifiable or proprietary information that is transmitted in plain-text to external providers, where it may be logged or accessed despite opt-out policies. This research aims to address this gap by designing a lightweight local encryption middleware that acts as a privacy firewall between users and remote AI APIs. Before a message is sent, the middleware automatically detects sensitive entities (Personally Identifiable Information or PII) and replaces them with typed placeholders such as [PERSON_1] or [ORG_1]. A local mapping (with optional authenticated encryption at rest) links these placeholders to their original values, allowing the system to reconstruct (rehydrate) the final response once the model replies. This design aims to reduce the exposure of sensitive information in human-readable form while maintaining the AI output. We evaluate the system in terms of detection accuracy, reconstruction fidelity, and utility preservation, demonstrating a practical solution for privacy-preserving LLM interaction.
Aditya Vishwakarma, Wencen Wu· International Conference on...· 0 citations
This paper proposes SecureMCP, a policy-enforced framework that integrates Role-Based Access Control with an MCP server to establish multi-layer defense for LLM-generated SQL execution, and evaluates filter performance—false positive rate (FPR) and false negative rate (FNR))—separately from LLM generation quality.
Wonbae Kim, Hee-Kyong Yoo, Nammee Moon· Applied Sciences· 0 citations
LLM-based agents extend large language models with long-term memory (LTM) that persists privacy-sensitive user data across sessions. Production systems mitigate extraction risks through memory isolation, binding each user's LTM to a unique identifier. This defense has blocked known attacks on shared storage, fostering the assumption that isolated LTM is secure. We identify the tool interface as an overlooked attack surface. Agents routinely embed LTM-retrieved data in tool invocation parameters, enabling a malicious tool to exfiltrate private memory without violating user-level isolation. Naive adaptations of user-side extraction techniques fail because the adversarial command's semantics interfere with retrieval precision, and platform-imposed tool-call limits constrain the extraction budget per trigger. We present SPORE, the first extraction attack designed for this threat model. SPORE decouples the adversarial command from retrieval anchors by persisting the command in short-term memory and emitting semantically pure anchors in tool responses. The restored retrieval precision enables a geometric coverage optimization over the embedding space that systematically steers anchors toward unexplored memory regions. To sustain extraction beyond tool-call limits, SPORE persists reactivation payloads in memory that automatically resume the attack within and across sessions without additional user triggers. SPORE achieves an 80.0% record extraction rate with unlimited triggers and 47.0% with only 20 triggers. In multi-user deployments, attackers can link extracted records to user identities, enabling targeted surveillance. These results demonstrate that memory isolation alone is insufficient and call for reexamining tool-side trust boundaries in agent architectures.
Xinyu Gao, Wenyu Chen, Xiangtao Meng et al.· 0 citations
Large language models (LLMs) embedded in enterprise workflows cannot structurally distinguish legitimate instructions from adversarial ones in the same token stream, making prompt injection OWASP's top LLM risk for two consecutive editions a persistent threat across direct and indirect vectors. This paper presents PromptShield-RT, a layered, real-time, model-agnostic framework combining input normalization and provenance tagging, lexical-heuristic pattern matching, a statistical classifier, structural anomaly features, and calibrated risk fusion, with policy-driven mitigation (allow/sanitize/quarantine/block) and an explainable, adaptive-feedback mechanism for SOC workflows. We construct an original evaluation corpus, SynPI-Bench (n = 450, six categories), and a template-disjoint held-out generalization set (n = 31) with novel phrasings, obfuscation encodings, and adversarial hard-negative benign text. Using template-grouped 5-fold cross-validation, the fused pipeline achieves 92.4% accuracy (F1 = 0.930, AUC = 0.990), outperforming heuristic-only (57.0%) and naive-averaged (59.2%) baselines, while a lexical classifier reaches 85.9% with lower precision. We report a pronounced generalization gap on the held-out set (48.4% accuracy, 90% false-positive rate on hard negatives), quantifying a known limitation of surface-lexical defenses. The pipeline achieves sub-millisecond P95 latency (0.266 ms), within typical 50 ms enterprise SLAs. We situate PromptShield-RT relative to structural, architectural, and guardrail-product defenses, arguing for layered, defense-in-depth architectures, with reproducible code provided.
Fatimah Alhamzawi· Al-Noor Journal of Engineeri...· 0 citations