These extensions focus on two approaches: curriculum learning and inference-time iterative feedback, where a Qwen-2.5-7B-Instruct model is used as a critic to provide corrective advice on previous round’s incorrect answers, allowing the model to learn from its own mistakes and make revisions.
This work proposes a self-evolving curriculum that, each round, evaluates the current checkpoint, seeds QbQ from the problems it can mostly get right, and trains on the resulting variants; under an identical data budget, this breaks the ceiling and lifts pass@1 to 16.5\% with no sign of saturation after 20 rounds.
Longtian Bao, Jianyou Wang, Yang Zhang et al.· 0 citations
It is shown that the transfer relationship between different difficulty levels characterizes the optimization dynamics induced by curriculum learning, which in turn explains the effectiveness of different curriculum schedules, and formalize this relationship as Relative Transfer, a principled measure of cross-difficulty knowledge transfer.
The first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing is conducted - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation.
Davide Romano, Kanak Raj, Jerrod Parker et al.· 0 citations
INSPIRE is an Internalize-Then-Improve approach combining Reference-Guided Student Internalization (RGSI), which produces high-quality preference candidates under the policy model's own distribution, with a stage-wise rubric preference training strategy that decomposes learning into method-oriented and correctness-oriented stages.
Shuai Wang, Jiayi Kuang, Yinghui Li et al.· 0 citations
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternatingandblocks:captures exploratory scratch work, whilecontains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces intoblocks and prompting an LLM to summarize each step into its corresponding. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08\% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each/pair, thecan be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85\% memory / context savings with an 8.67\% performance drop across mathematical tasks.
Zain Sarwar, Supriyo Chakraborty, B. Kapusuzoglu et al.· 0 citations
This work presents a theoretical framework that reveals how reasoning steps can amplify error through three failure modes: incorrect sub-task decomposition, incorrect sub-task solving, and incorrect final answer summarization, and introduces structured interventions that adapt CoT generation according to the identified failure types.
Haibo Jin, Peiyan Zhang, Man Luo et al.· Neural Information Processin...· 1 citation