Preprint

Human-Machine Collaboration on Generative Meta-Learning: Model and Algorithm

M. P. Unni Samuel Kaski

Jul 2026 · 0 citations · 22 references

Computer Science

TL;DR

Empirical validation shows that GMHF substantially reduces deployment loss as expert reliability increases, and that the divergence between generated and target data falls under reliable feedback, directly corroborating the divergence-minimisation mechanism predicted by the theory.

Abstract

Generalizing machine learning models to environments that differ from their training distribution remains a critical hurdle, particularly when data from the target domain is entirely or partially unavailable. We propose Generative Meta-Learning with Human Feedback (GMHF), a novel framework that bridges this domain gap by leveraging expert intuition to guide data synthesis. Grounded in a theoretical analysis of generalization error, we derive bounds demonstrating that aligning the distribution of generated data with human beliefs regarding the target physics significantly mitigates risk. GMHF operationalizes this insight by employing a Conditional Neural ODE (cNODE) as a generative digital twin, coupled with a Reinforcement Learning (RL) agent. The agent iteratively refines the latent physical parameters of the generated trajectories based on feedback, effectively steering the meta-learner toward the unobserved target distribution. Empirical validation on a nonlinear Duffing oscillator shows that GMHF substantially reduces deployment loss as expert reliability increases, and that the divergence between generated and target data falls under reliable feedback, directly corroborating the divergence-minimisation mechanism predicted by our theory. Further experiments on a non-dynamical probabilistic model confirm that the framework extends beyond ODE-governed systems, establishing human-AI collaboration as a rigorous catalyst for robust generalisation under distribution shift.

arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

Learning Generalizable Behaviors for Terminal Agents

River, a simple training recipe that improves reward quality by filtering low-quality environments and augmenting outcome rewards with process-level behavior regularization is proposed, which achieves the best performance among evaluated open-source RL-trained 8B models across four terminal-agent benchmarks.

Yi-Fan Yao, Bo Pang, Xuan-Phi Nguyen et al. · 0 citations

#artificial intelligence Preprint Aug 2026

Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL

This work introduces Co-RL, a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers, and shows that unsupervised reasoning can emerge through cooperative multi-agent training.

Yunhao Yang, Yuexin Bian, Yunjie Tian et al. · 0 citations

VEG: Verbal ϵ -greedy for Semantic Exploration in Multi-Turn RL Agents

This work proposes VEG (verbal ϵ -greedy), a novel framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space and achieves superior accuracy compared to standard RL baselines.

Yongchang Hao, Jie Hao, Yongsheng Mei et al. · 0 citations

Preprint Jul 2026

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

MeRLa (Meta-Learned Reward Shaping), a principled framework that meta-learns a task-aware shaping function across auxiliary tasks before RLHF training, is introduced, providing theoretical guarantees for policy invariance, analyze representation drift sensitivity, and formally address incentive misalignment from entropy maximization.

Yu-An Chu · 0 citations

Preprint Jul 2026

Learning Latent Reasoning Traces for Scalar Reward Models End-to-End

LatentRM is a reward modeling framework that learns intermediate reasoning traces as discrete latent variables to explicitly maximize the likelihood of downstream scalar rewards through on-policy optimization of the latent reasoning space end-to-end.

Sanwoo Lee, Clive Bai, Hsiu-Yuan Huang et al. · 0 citations

#machine learning Preprint Aug 2026

Demystifying Reinforcement Learning Post-Training of Language Models

This work deconstructs the RL post-training algorithm, investigating each step to clarify what is actually happening beneath the surface, and uses the entropy of the policy's output distribution as a lens to compare the distributions learned through pretraining, SFT, and RL post-training, revealing how each stage shapes model certainty.

D. Clay, Saket Gollapudi, Sankar V Harilal et al. · 0 citations