A Local Middleware for Privacy-Preserving LLM Inference via Reversible Entity Substitution

Abstract

Large Language Models (LLMs) such as ChatGPT, Gemini, Claude, etc. have become integral tools for technical writing, software development, and communication. However, these systems, despite privacy assurances, still expose a significant privacy risk. User prompts often contain personally identifiable or proprietary information that is transmitted in plain-text to external providers, where it may be logged or accessed despite opt-out policies. This research aims to address this gap by designing a lightweight local encryption middleware that acts as a privacy firewall between users and remote AI APIs. Before a message is sent, the middleware automatically detects sensitive entities (Personally Identifiable Information or PII) and replaces them with typed placeholders such as [PERSON_1] or [ORG_1]. A local mapping (with optional authenticated encryption at rest) links these placeholders to their original values, allowing the system to reconstruct (rehydrate) the final response once the model replies. This design aims to reduce the exposure of sensitive information in human-readable form while maintaining the AI output. We evaluate the system in terms of detection accuracy, reconstruction fidelity, and utility preservation, demonstrating a practical solution for privacy-preserving LLM interaction.