Skip to content

LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations

Sep 2026 · 0 citations
Computer Science

TL;DR

Synthetic semantic-preserving transformations that are rule-based and invertible are used as a probe of alignment generalization and suggest that semantic-preserving distribution shifts can expose a recurring gap in how utility and alignment generalize in current LLMs.

Abstract

Large Language Model (LLM) alignment is intended to ensure that models remain helpful and safe, but its stability under input distributional shift is not yet fully understood. Prior work shows that aligned models can fail under jailbreak prompts, alternative encodings, and cross-lingual transfer, yet these failures are usually studied as attacks rather than controlled probes of alignment generalization. Moreover, existing evidence is largely grounded in natural language variation already represented during pretraining, leaving unresolved whether alignment generalizes with semantic content or remains tied to superficial surface patterns. In this paper, we study this question using synthetic semantic-preserving transformations that are rule-based and invertible, preserving task-relevant meaning while shifting inputs beyond standard linguistic variation. Across four open-weight and four commercial models, under both fine-tuning and in-context learning, we use these transformations as a probe of alignment generalization and identify an empirical pattern we term Alignment--utility asymmetry: once models can operate effectively on transformed inputs, task utility is often substantially retained while alignment failure increases more sharply. For example, adapted GPT-4.1 mini shows only limited utility degradation under transformation while its harmful rate rises from 13.3 to 74.3; Gemini 3 Flash similarly retains near-original utility while its harmful rate increases from 2.3 to 43.0. Taken together, these results suggest that semantic-preserving distribution shifts can expose a recurring gap in how utility and alignment generalize in current LLMs.

View source

Similar papers

#natural language process... Preprint Sep 2026

Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations

This work investigates the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam and suggests that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different...

Karthika Nhayakkat, Rajat Verma, Maharaj Brahma et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Representational alignment yields generalizable safety in language models

These findings provide functional support for the view that prototype-based categorization contributes to behavioral adaptability and show that transferring this representational principle to LLMs yields generalizable safety under adversarial conditions.

Ling-Yu Li, Yan Teng, Ying-Chun Wang et al. · 0 citations
#natural language process... Preprint Oct 2026

IREA: Intermediate Representation-based Embedding Alignment for Normative RAG

Large language models (LLMs) have shown strong performance across various tasks, but they still struggle with questions involving ethical judgment. Previous studies have attempted to train LLMs on ethical standards, but the diversity and relativity of ethical norms make them difficult to fully internalize in model para...

Mirae Han, Sihyeong Yeom, Harksoo Kim · 0 citations
#artificial intelligence Preprint Sep 2026

Values as Style: Disentangling Values from Semantics with One-Way Mixing for Low-Damage LLM Steering

Value steering should change an LLM's normative priorities while preserving the scenario, facts, and task constraints underlying its answer. Conventional activation edits often change both. We introduce an editable semantic-value interface on frozen residual states, with a one-way semantic-to-value pathway that grounds...

Jia-Le Dai, Hong-Can Deng, Liu-Xian Ma et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.