This work finds that models cannot reliably reconstruct the original meaning and layer-wise probe classifiers fail to detect the harmful intent of perturbed prompts, and perturbations can occasionally reduce attack success by inducing off-topic or incoherent responses.
Attention Diversion explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time.
Jiaqi Zhu, Yang Zhang, Junhua Ding et al.· 0 citations
Recent jailbreak attacks demonstrate that large language models (LLMs) can be manipulated to generate harmful outputs through adversarial prompts even after robust alignment. However, prevailing methods typically focus on forcing a desired response at the output layer—a surface-level strategy that is brittle and often fails to bypass the more fundamental safety checks embedded within the model’s internal mechanisms. In contrast, we propose intermediate consistency greedy coordinate gradient (IC-GCG), an approach that extends the GCG framework to explicitly target the model’s internal alignment mechanism. Our method forces a harmful prompt’s hidden representations to mimic those of a semantically related benign counterpart within critical middle layers. By adapting a greedy coordinate optimization strategy to craft an adversarial suffix that minimizes this representational distance, IC-GCG creates “semantic camouflag” that bypasses the model’s internal safety judgments. Experiments on several aligned LLMs indicate that IC-GCG substantially outperforms state-of-the-art baselines, increasing the average attack success rate to 94.3% and proving particularly effective against strongly aligned models such as LLAMA2-7B-CHAT, where it boosts success by over 56 percentage points. Furthermore, the resulting suffixes display superior transferability to other models. Ablation studies confirm that this high efficacy critically depends on a strong semantic link between the harmful and benign prompt pair, empirically validating that the integrity of intermediate representations is a crucial vulnerability and highlighting the need for a new class of defenses that operate directly on the model’s internal states.
Zichu Ren, Donghai Zhu, Haibo Hong et al.· IEEE Internet of Things Jour...· 0 citations
It is shown that the strength of this signal depends on text length and sampling range: spectral evidence is clearest for long, continuous, constrained generation, while short, fragmented, mixed, and edited settings require complementary confidence and fluctuation views.
Haitong Luo, Xuying Meng, Weiyao Zhang et al.· 1 citation
This work proposes TinyAttack, a novel adversarial attack framework designed to exploit vulnerabilities in LLMs through Unicode-based stylistic transformations, which utilises five Unicode variants to modify the visual rendering of text without altering its underlying semantic or syntactic structure.
M. Mamta, B. Grecu, O. Cocarascu· Annual Meeting of the Associ...· 0 citations
Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. Nevertheless, sentence completion remains vulnerable to incomplete harmful prompts. In this work, we formalize this phenomenon as incomplete prompt jailbreaks (IPJ) and provide a systematic empirical characterization of when and how incomplete prompts elicit harmful continuations. We analyze diverse attractor types associated with incomplete sentence continuation and show that LLMs systematically delay refusal until sentence termination. We further demonstrate that training models to refuse incomplete harmful prompts via parameter tuning is insufficient, failing to generalize across both content domains and attractor types. To enable fine-grained control, we identify two functional neurons: termination and continuation neurons. By clarifying their roles in sentence completion, we highlight the potential of neuron-level interventions for more precise and robust IPJ defenses.
Y. Kim, Bumjin Park, Jaesik Choi· Annual Meeting of the Associ...· 0 citations
This work investigates the impact of cross-lingual prompt injection attacks on LLM-based relevance judgments using TREC Deep Learning collections and two open-weight models under established prompting frameworks, and demonstrates that multilingual query-based injections are highly effective in inflating relevance scores while simultaneously evading existing prompt-injection defenses.