#artificial intelligence #machine learning #natural language processing Preprint

Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning

AR Becker Jakob Kemmler David Thulke Christine Schäfer C. Dugast Hermann Ney

Aug 2026 · 0 citations · 48 references

Computer Science

TL;DR

Experiments show that knowledge-aligned SFT can reduce factual hallucinations on WildHalu and Biography while largely preserving general capabilities and confirm that SFT targets beyond the base model's knowledge drive hallucination behavior.

Abstract

Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We study this as a source of hallucinations and frame a group of mitigation methods as \emph{knowledge-aligned SFT}: constraining SFT training targets to the base model's parametric knowledge. Under a unified setup, we compare existing generation-based and estimation-based knowledge-alignment methods and introduce two new variants: Evidence Rewrite, which verifies base-model generations using external evidence, and Recall Rewrite, which retains claims only when they can be consistently recalled by the base model. Experiments with Qwen 3 4B and OLMo 3 7B show that knowledge-aligned SFT can reduce factual hallucinations on WildHalu and Biography while largely preserving general capabilities. Recall Rewrite yields the strongest factuality gains and improves refusal behavior on UnknownBench. It thereby confirms that SFT targets beyond the base model's knowledge drive hallucination behavior.

DOI arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

ProtoLIP: From Sentence-Level to Object-Level Evidence Disentanglement

To disentangle visual evidence at both semantic and spatial levels, ProtoLIP is introduced, a lightweight prototype-mediated evidence layer that organizes reusable visual prototypes into text-derived semantic families and uses coarse-to-fine evidence routing, where semantic families constrain prototype eligibility and...

Yan Zhu, Yong-Bo Chen, Zheng-Ming Ding et al. · 0 citations

#artificial intelligence Preprint Sep 2026

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations. Finetuning methods such as RAFT and PA-RAG enhance...

Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi et al. · 1 citation

#artificial intelligence Preprint Sep 2026

Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models

Modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask, suggests modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.

Gautam Rajendrakumar Gare, Si-Ying Li, He-Wei Wang et al. · 1 citation

#artificial intelligence Preprint Sep 2026

Probing Factual Knowledge Transfer with Training Data Interventions

The results show that fact transfer is very limited: under the strictest removal condition, a large majority of English-acquired facts fail to transfer into Persian, and it is shown that sentence-level co-occurrence removal is insufficient to eliminate fact signal.

Romina Oji, Marc Braun, Marcel Bollmann et al. · 0 citations

Preprint Aug 2026

Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO

Refusal-Calibrated Group Relative Policy Optimization (RC-GRPO) is proposed, a calibrated RL strategy that strengthens the refusal ability of MLLMs while preserving localization performance, and enforces"None" outputs in rollouts for valid advantage estimation on negative samples and applies a penalty to prevent over-r...

Xuzheng Yang, Jun Ling, Tao Huang et al. · 1 citation

#natural language process... Preprint Sep 2026

Do LLMs Make More Mistakes If They Do Not Believe the Input Data?

How faithfulness of LLMs to provided context depends on how plausible they perceive the context to be (context-memory conflict) is analyzed, which makes use of the increased difficulty of non-English and low-resource language text generation and input data based on local knowledge.

Peter Kochelka, Aleš Manuel Papáček, Vojtěch Dvořák et al. · 0 citations