Experiential abstractions improve LLM performance on mathematical and logical reasoning benchmarks, and the abstraction usage framework can transfer to other datasets and models.
Abstract
Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more effectively. We study whether Large Language Models (LLMs) can similarly benefit from such experiential abstractions. From LLMs'solution traces on the MATH training set, a stronger teacher or the LLMs themselves extract natural-language abstractions into a retrievable library. We explore two usage modes: (1) inference-time retrieval and (2) reinforcement learning (RL) with abstraction-augmented training prompts. Experiential abstractions improve LLM performance on mathematical and logical reasoning benchmarks. Self-extracted abstractions match teacher-extracted ones, and our abstraction usage framework can transfer to other datasets and models. These findings suggest LLMs can extract and apply experiential abstractions much as humans leverage distilled experience.
This study studies how LLMs learn from iterative experience at test time, a setting the authors refer to as Chain-of-Experience (CoE), where models accumulate experiential traces through iterative interactions with self or environmental feedback to form a continual improvement loop beyond zero-shot inference.
Hao-Qin Tu, Yun-Hao Fang, Yizhong Wang et al.· 0 citations
This work proposes DeepInstructor, an agentic framework that formulates idea evaluation as reasoning over structured scholarly experience, and introduces DeepInstruct, a dataset with controlled pairwise comparisons across novelty, significance, and feasibility.
Rong-Can Pei, Fang Guo, Qing-Lin Qi et al.· 0 citations
This work proposes Learning to Coach (L2C), a framework that trains a dedicated LLM-as-a-Coach to extract actionable experiential knowledge from an actor model's previous trajectory, and studies two such rewards: a same-instance reward, which improves subsequent responses on the original problem, and a cross-instance r...
Guan-Heng Chen, Tian-Zhu Ye, Li Dong et al.· 0 citations
Large language models (LLMs) are capable of completing a variety of tasks, but remain unpredictable and intractable. Representation Control (RepControl) seeks to resolve this problem through targeted interventions that modify high-level representations of concepts such as honesty, harmfulness or power-seeking. We forma...
It is found that models generally learn more reliably from rules than from examples alone, and additional examples on top of rules or simply scaling up the number of examples do not lead to consistent and significant gains.
Xiang Fu, Seungmin Cho, Yukyung Lee et al.· 0 citations
Echo-GRPO is proposed, a framework that lets the model reason in the words it speaks, and idiolectal paraphrasing is a plug-in module that consistently improves both RL and supervised fine-tuning frameworks for reasoning distillation, demonstrating that policy-aligned supervision extends beyond GRPO.
Ji Soo Lee, Jinyoung Park, Seohyun Lee et al.· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comOct 22, 2026
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
Microsoft Research Blog· microsoft.comOct 15, 2026
Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
