Skip to content

Your Model Is Leaking: Covert Information Transfer through LLM Residual Streams

Sep 2026 · arXiv.org · Vol abs/2609.27996 · 0 citations · 52 references
Computer Science

TL;DR

A residual-stream covert-channel attack is introduced that maps messages to codewords and injects them into an intermediate residual stream through a compromised runtime hook and is scaled with the local residual norm using the signal-to-residual-norm ratio.

Abstract

Privacy-sensitive organizations may run large language models (LLMs) in restricted or air-gapped environments while exporting selected diagnostic artifacts. We show that a compromised runtime component can hide sensitive information in intermediate activations that are allowed to leave the restricted environment. An offline observer can recover this information with a simple linear decoder. The attack requires no model retraining or weight modification, no attacker-controlled egress, and no control over the recorder or transfer process. We introduce a residual-stream covert-channel attack that maps messages to codewords and injects them into an intermediate residual stream through a compromised runtime hook. To maintain recoverability, the injection strength is scaled with the local residual norm using the signal-to-residual-norm ratio. Across eleven models from seven architecture families, our evaluation shows 91--100% recovery on nine models with KL divergence 0.001--0.007, while evaluated activation-level detectors remain close to random guessing (AUC<= 0.56). Tested post-hoc defenses do not reliably eliminate the channel. Thus, an activation artifact can be schema-valid while carrying information that is not authorized to cross the boundary.

View source

Similar papers

#machine learning Preprint Sep 2026

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

A systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while leaving an observable channel untested, but the system is not thereby safe: five classes of attack, including those accumulating observations across training steps, were never measured.

G. Politis, E. Pappas · 1 citation
#machine learning Preprint Oct 2026

When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry

Mixture-of-Experts (MoE) language models produce routing information during inference that may be logged or exposed for monitoring, debugging, load analysis, and safety auditing. Unlike ordinary model outputs, this telemetry reveals a view of the model's internal computation, raising a privacy question: can it reveal w...

Yi-Xin Tan, Jia-Yang Liu, Lu-Peng Sun et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokeni...

Roy Weiss, B. Konstantinov, Eitam Sheetrit et al. · 0 citations
Preprint Aug 2026

Inadvertent Context Leakage in Language Models

Leakage enables two practical attacks: a trained classifier that infers semantic predicates about user memories from routine natural-language outputs, and an RL-trained adversary that extracts full Social Security Numbers from a production-style agent.

Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri et al. · 0 citations
#artificial intelligence Preprint Sep 2026

CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents

CIPL (Channel Inversion for Privacy Leakage), a channel-aware evaluation framework for black-box privacy leakage in LLM agents, is presented and provides a common framework for comparing how internal sensitive dependence is realized as externally recoverable leakage across heterogeneous agent pipelines.

Tao Huang, Guo-Xin Wu, Guo-Long Zheng et al. · 0 citations
Preprint Oct 2026

Anytime-valid detection of LLM weight exfiltration

A compromised LLM inference server can leak model weights by encoding payload bits in otherwise plausible token choices. A replay of the same prompt in a trusted server can expose such deviations, but benign numerical nondeterminism also causes token mismatches. Patient attackers can therefore hide within normal variat...

Ines Ortega-Fernandez, Mateusz Kowalczyk, Keri Warr · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.