Review

A Survey of Joint Online-Offline Fine-tuning for Large Language Models

Taihang Zhen Guang Yang Chenzhang Li Nuoyu Yan Shilong Zhou Guangyu Liu Xiaotong Tang Jing Huo Boyan Wang Junlan Feng Yuyao Zhang

· 0 citations · 41 references

TL;DR

This work advocates for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT, and provides the first comprehensive survey focusing specifically on the synchronization of data provenance.

Semantic Scholar

View source

Similar papers

Preprint Aug 2026

Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes

Evaluation-Conditioned Training (ECT), a post-training framework that uses natural language to condition each training sample on the fidelity of the feedback the authors provide and then elicits the desired behavior by conditioning the LLM on a high-fidelity monitor in deployment, is introduced.

Alec Harris, Kasey Corra, Archie Chaudhury et al. · 0 citations

Preprint Jul 2026

Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

Large language models (LLMs) remain expensive to fine-tune because full-parameter updates require substantial memory, compute, and per-task storage. We study whether saliency signals originally developed for pruning can be reused to choose where a model should adapt. We propose Super, a sparse parameter-efficient fine-tuning (PEFT) method that fixes a small trainable support using a Wanda-style activation-weighted magnitude score [Sun et al., 2023] computed from a calibration pass. We then introduce Supra, a hybrid adapter that combines this sparse update with LoRA while preserving a matched trainable-parameter budget through a simple budget-splitting rule. In single-seed Math17K arithmetic experiments on Llama-3.2-1B and Meta-Llama-3-8B, the best Super/Supra variants achieve the highest average accuracy among the tested schedule-selected adapter configurations. We also include a PaFi-style magnitude-only support as a closest training-free sparse baseline and find that low-score supports under both magnitude and Wanda-style orderings can be effective. These results suggest that simple pruning-inspired orderings can provide useful fixed sparse supports for PEFT, especially when combined with low-rank adapters.

Ivan D. Ilin, Philip Zmushko, Peter Richtárik · 0 citations

Preprint Jul 2026

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

This work proposes a three-phase pipeline that resolves this trilemma by decoupling syntax acquisition from algorithmic reasoning and applies Reinforcement Learning with Verifiable Reward~(RLVR) grounded by language-agnostic Input/Output tests.

D. Samaraweera, Anjana Supun, Srinath Perera · 0 citations

Preprint Aug 2026

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

A simple modification to supervised fine-tuning, TailSFT, which filters out already fit sequences during training, thereby focusing learning on under-modeled regions, or the tail, of the data distribution, and introduces a lightweight diagnostic for identifying settings where TailSFT is most likely to help.

Sadhika Malladi, Samy Jelassi, Dylan J. Foster et al. · 0 citations

Preprint Jul 2026

REPREC: Representation Driven Parameter-Efficient Recommendation System

REPREC is a lightweight framework that conditions a frozen LLM using compact user-level representations and consistently improves recommendation performance across different sequential encoders, LLM backbones, and user activity levels.

Harshini Kavuru, Dwipam Katariya, Giri Iyengar et al. · 0 citations

Preprint Jul 2026

GFlowRL: Scaling Distribution-Matching RL to Large Language Models

GFlowRL, a streamlined GFlowNet-style RL algorithm that removes the auxiliary partition network entirely while preserving the reward-distribution-matching objective, is proposed, and is the first GFlowNet-style RL algorithm to scale stably across both dense and sparse architectures.

Xiaodong Liu, Michael Xu, Jack W. Stokes et al. · 0 citations