Skip to content

Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?

Sep 2026 · arXiv.org · Vol abs/2609.35868 · 0 citations · 27 references
Computer Science

TL;DR

Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings, inspired by the role of activation gradients in local risk reduction, targets useful adaptation updates rather than source-text reconstruction or linguistic fluency.

Abstract

Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form. We propose Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings. Inspired by the role of activation gradients in local risk reduction, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency. The resulting embeddings are used directly for downstream fine-tuning; discrete token projections are employed only for qualitative inspection. Experiments on six models from the Llama and Qwen families, ranging from 1B to 32B parameters, cover six benchmarks spanning knowledge, mathematical reasoning, code generation, and commonsense reasoning. Under matched LoRA adaptation settings, DASA achieves performance comparable to the source natural-language data and surpasses it in multiple configurations, while outperforming GRADMM in most comparisons. Further experiments cover general-domain and task-specialized source data. Under the evaluated synthesis settings, DASA provides a $3.6$--$4.9\times$ speedup over GRADMM with comparable peak GPU memory.

View source

Similar papers

#natural language process... Preprint Sep 2026

Generating Adversarial Texts for Machine Translation via GRPO

This work proposes a scalable reinforcement-learning-based approach for rewriting existing source texts into instances that are more difficult to translate for MT systems, using a large language model with Group Relative Policy Optimization (GRPO) and fine-tune a large language model with Group Relative Policy Optimiza...

Florian Zogaj, Jakob Hütteneder, Giovanni De Muri et al. · 1 citation
Aug 2026

Augmenting Text to Increase Translation Difficulty

This work proposes augmenting existing benchmarks to increase translation difficulty by combining adversarial optimization with a differentiable translation difficulty estimator, and uses gradients from a combined difficulty and fluency objective to iteratively replace tokens in Adversarial Translation Optimization (AT...

William Kalikman, Šimon Sukup, Michal Tesnar et al. · 2 citations
#artificial intelligence Preprint Oct 2026

Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models

Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale. We close t...

Marjan Celikik, A. Ramallo, Javier Morales · 0 citations
#natural language process... Preprint Sep 2026

Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

This work studies how to make LLMs natively generate TTS-friendly text, which is frame as a preference alignment problem: instead of relying on downstream rewriting modules, this work directly align LLMs to generate text optimized for spoken delivery.

Thibaut Thonet, Jos Rozen, Laurent Besacier · 0 citations
Preprint Aug 2026

From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation

By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cro...

Lukas Edman, Daryna Dementieva, Alexander Fraser · 0 citations
#artificial intelligence Preprint Sep 2026

Prompt-Robust Language Models: Which Training Strategies Work?

The current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance, and recent robustness-enhancing methods often fail to outperform the simplest data construction strategy: training on one template per batch.

Frederic Sadrieh, Michal Štefánik · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.