This work presents a systematic mechanistic analysis of multilingual safety using sparse autoencoder features, sparse interpretable directions in the residual stream associated with harmful and harmless model behavior across three instruction-tuned LLMs, eight languages, and all model layers to qualify the language-universality of safety alignment as architecture-dependent and offer a mechanistic account of multilingual safety interventions.
Abstract
Safety alignment of large language models (LLMs) degrades across languages, yet the internal mechanism driving this asymmetry remains poorly understood. Our work, therefore, presents a systematic mechanistic analysis of multilingual safety using sparse autoencoder (SAE) features, sparse interpretable directions in the residual stream associated with harmful and harmless model behavior across three instruction-tuned LLMs, eight languages, and all model layers. We observe that safety-relevant features are architecture-dependent in terms of where they are located and how they are distributed across layers. Additionally, they are geometrically entangled with language identity and exhibit cross-lingual sharing patterns, i.e., languages share safety features to varying degrees across model depths and architectures. This safety-language entanglement has direct consequences such that ablating safety features impacts not only harmful response rates but also target language, with the degree of intervention predicted by the relationship between safety and language features. Our findings qualify the language-universality of safety alignment as architecture-dependent and offer a mechanistic account of multilingual safety interventions.
This work investigates cross-lingual safety transfer in four African languages, Twi, Hausa, Amharic, and Swahili, using LoDNA, a new safety dataset that pairs literal translations with culturally localized prompts to demonstrate superficial safety alignment.
Abigail Oppong, P SAM SAHIL, Tadesse Destaw Belay et al.· 1 citation
Uncovering the internal mechanisms underlying the safety capabilities of large language models (LLMs) is crucial for developing trustworthy artificial intelligence. Currently, mechanistic interpretability studies on multilingual safety are largely confined to local components, such as isolated neurons. However, this st...
Shu-Yi Miao, Wang-Jie Qiu, Pengyang Shao et al.· 0 citations
This survey provides the first comprehensive review of research on code-switched LLM safety and robustness evaluation, and aims to guide researchers and practitioners in developing more robust evaluation frameworks and alignment techniques for real-world multilingual deployments.
Pulagam Naveen Kumar, Sowjanya Bojja, K. Anoosha et al.· International Journal for Re...· 0 citations
Safety alignment in large language models is overwhelmingly trained on English data, yet models are deployed across typologically diverse languages. We evaluate six open‐weight large language models (LLMs) across 17 bilingual Korean/Chinese prompt conditions and three translation engines (NLLB‐200, Gemini 3.0 Pro, an...
The findings suggest that activation steering may provide a practical, low- cost mechanism for extending English-derived safety signals to other languages, and introduce a multilingual translation-and-evaluation pipeline to facilitate future work on cross-lingual safety interventions.
Emma V. Stein, Dominik Meier, T. Ruas et al.· 1 citation
MMSAFE is proposed, a multi-layer framework for multilingual safety-degrading data identification that captures both shared and language-specific safety signals and reduces the average harmful-response ratio compared with random filtering and achieves stronger average performance than the strongest single-layer baselin...
Jia-Kun Li, Guo-Wei Song, Si-Jia Li et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.