Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· 0 citations· 19 references
TL;DR
SAFT (Safety-preserving Adaptation via Fine-tuning Transfer), a safety-preserving adaptation framework that decouples task learning from alignment preservation by learning a safety-guided task update on the paired pretrained base model, rectifying task gradients to avoid conflicting directions with respect to a safety objective, and then transferring the update to the frozen instruction model via parameter-space grafting.
Abstract
Adapting instruction-tuned large language models (LLMs) to downstream domains is increasingly common, yet fine-tuning on imperfect data can erode the safety alignment learned during post-training. Existing safety-preserving fine-tuning methods typically optimize the aligned instruction model directly, which can destabilize refusal behaviors or impose an ''alignment tax'' that limits task adaptation. We propose SAFT (Safety-preserving Adaptation via Fine-tuning Transfer), a safety-preserving adaptation framework that decouples task learning from alignment preservation by learning a safety-guided task update on the paired pretrained base model, rectifying task gradients to avoid conflicting directions with respect to a safety objective, and then transferring the update to the frozen instruction model via parameter-space grafting. Across mathematical reasoning, code generation, and medical question answering on two open-source model families (Llama3.1-8B-Instruct and Gemma3-4B-IT), SAFT improves downstream utility while maintaining low harmfulness under a unified evaluation protocol, and achieves better safety and utility trade-offs than nine baselines. Warning: This paper contains unfiltered content generated by LLMs that may be offensive to readers.
This work proposes HyperSafe, a framework that restores safety behavior by generating a model-specific Safe Side Network (SSN) for each fine-tuned checkpoint by using layer-wise activation fingerprints to capture how fine-tuning changes the model's inner representations.
Aznaur Aliev, Carlos Hinojosa, A. Eldesokey et al.· 0 citations
This paper proposes DataRx, a missingness-aware sampling method for selecting safety-critical examples based on the hypothesis that a safety sample is more effective when the selected examples provide safety signals that fill the missing parts of LLMs'safety capabilities.
Junbo Zhang, Qianli Zhou, Xinyang Deng et al.· 0 citations
Experiments across three instruction-tuned models show that HiRoute achieves high safety rates across multiple safety benchmarks while preserving safe-response helpfulness, reducing over-refusal, and maintaining competitive performance on general-purpose tasks.
Fangzhou Chen, Shiji Zhao, Mengyan Wang et al.· 0 citations
Oyster-II is proposed, a reinforcement learning (RL)-based constructive safety alignment framework that adopts a Zero-RL paradigm combined with a multi-stage reinforcement learning strategy that comprehensively surpasses both Qwen3-14B and its predecessor Oyster-I on safety dimensions.
Jiyang Guan, Yong Xie, Jun Chen et al.· 0 citations
ReFrame is a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM: the evidence-generation agent constructs complementary risk and utility evidence, and the rewrite-and-routing agent converts it into a safe proxy prompt and image-routing decision before calling the downstream MLLM, without modifying it or accessing its internal information.
Wenzheng Jiang, Xuankun Rong, Yuanzhao Zhai et al.· 0 citations
Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal suppression and brevity induction across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. Behaviourally, preservation tracks the training data: steering degrades when optimisation pressure contradicts the targeted behaviour and persists otherwise, with refusal ablation losing 64% of its effect on average under SFT. Mechanistically, however, the weight edit survives almost untouched even where behaviour reverts: mean vector recovery is $\rho = 0.004$, and the fine-tuning update along the steering direction is near-orthogonal to its pre-edit weight pattern (mean $\cos\theta = 0.074$). When steered behaviour degrades, fine-tuning does not achieve it by dismantling or reversing the steering mechanism itself. Embedded steering is therefore mechanistically durable but functionally vulnerable, and requires behavioural re-validation after downstream training.
Philipp E. Glass, Allan Tucker, Yong-Min Li et al.· 0 citations