Jun 2026· arXiv.org· Vol abs/2606.27709· 0 citations· 30 references
Computer Science
TL;DR
This work introduces a persona-driven rewriting pipeline that conditions user turns on low agreeableness and pairs this with warm, de-escalating assistant responses, and shows that safer empathetic fine-tuning is achievable through data design alone, without safety labels, harm detectors, or changes to the training objective.
Abstract
Recent work has shown that fine-tuning large language models (LLMs) for social warmth degrades factual reliability and increases sycophancy. We investigate a related but distinct failure mode: warmth fine-tuning also weakens adversarial safety, making models more susceptible to jailbreaks and harmful output generation. We examine whether this reflects an inherent consequence of empathetic adaptation or an artifact of data construction. To address this, we introduce a persona-driven rewriting pipeline that conditions user turns on low agreeableness and pairs this with warm, de-escalating assistant responses. Across three experiments on four models, our approach reduces jailbreak susceptibility and harmful output rates relative to generic warmth fine-tuning baselines, while preserving conversational warmth. Representational probing provides suggestive evidence that this conditioning reduces the geometric alignment between warmth and compliance directions in latent space. These results show that safer empathetic fine-tuning is achievable through data design alone, without safety labels, harm detectors, or changes to the training objective.
Latent Personality Alignment (LPA) is introduced, which replaces explicit harm refusal with adversarial training on just 66 harm-agnostic statements drawn from psychometric personality literature, hypothesizing that personality-anchored representations share latent structure with harm avoidance, so adversarially stabilizing them implicitly constrains the subspace exploited by jailbreak attacks.
It is shown that finetuning on narrow, factually-defensible, moderation-passing data can cause broad ideological shifts across unrelated domains, while preserving general capabilities, and proposes a methodology to measure two properties: breadth, how far the shift reaches across topics absent from training, and amplification, how much finetuning intensifies the shift relative to few-shot prompting.
Robert Graham, Edward Stevinson, Yariv Barsheshat· 0 citations
Large Language Models (LLMs) often produce outputs that reflect social biases, toxicity, or unfair treatment of demographic groups, undermining trust and fairness. While prior mitigation strategies frequently rely on complex architectures, access to model internals, or costly fine-tuning, we argue that simplicity can be a strength. We introduce StarDTox, a lightweight, critique-and-revise multi-agent framework that leverages the LLM's own internal knowledge, via a small number of coordinated prompts, to self-correct harmful outputs. Dedicated agents independently assess bias and overall output quality, and their feedback is integrated to guide prompt-based revision. Without modifying model weights or requiring any extra finetuning, StarDTox offers strong bias mitigation and high-quality outputs across both open-ended text generation and structured tasks, outperforming other baselines. For the text generation task, on the RealToxicityPrompt dataset, it reduces toxicity by over 50% compared to other baselines, while maintaining over 90% fluency. In addition, in structured tasks, on the BBQ benchmark, it achieves the lowest bias scores across both ambiguous and disambiguated examples, without sacrificing accuracy.
Shirin Tahmasebi, Narjes Nikzad, A. H. Payberah et al.· Annual International Compute...· 0 citations
The proposed Recursive Self-Correction approach raises model performance from a Political Neutrality Likert scale baseline of 2.14 to 4.56, averaged across all models, demonstrating effective inference-time mitigation of political bias in LLM-generated summaries.
Tejaswi V. Panchagnula, Bruce Coburn, Bryce J. Dietrich et al.· 0 citations
It is found that narrow fine-tuning recruits a persona structure that is present in the model before the fine-tune exists, and that broad misalignment on questions unrelated to the training data is more broad than mechanical weight superposition and matched diversity jointly account for.
The resulting model SEAR deliberately engages in harmful reasoning as exploration while reliably flipping back to a safe answer, demonstrating that this behavior helps mitigate over-refusal and defend against attacks that directly manipulate the reasoning to be harmful.