Skip to content

In-Context Guidance: Learning Inter-Task Synergies via Numerical Foundational Models for Few-Shot Multitask Optimization

Sep 2026 · arXiv.org · Vol abs/2609.25836 · 0 citations · 43 references
Computer Science

TL;DR

Experiments across synthetic benchmarks and a real-world robot arm control problem, together with evaluations under different acquisition functions and evolutionary multitasking, demonstrate the effectiveness and generality of ICG-MTO for few-shot multitask optimization.

Abstract

Multi-task optimization (MTO) addresses a set of optimization tasks simultaneously, often suffering from inaccurate inter-task relationship estimation under limited evaluation budgets, leading to negative transfer. This paper introduces In-Context Guidance Multitask Optimization (ICG-MTO), a novel framework that leverages numerical foundational models to improve inter-task coupling estimation in few-shot scenarios. Unlike conventional methods that rely solely on scarce observed data, ICG-MTO employs a frozen foundational model to infer auxiliary guidance through in-context learning. The framework operates through three stages: constructing an algorithm-specific in-context query from evaluated solutions, using the foundational model to infer a guidance signal characterizing predictive relationships among tasks, and translating this signal into algorithm-specific guidance for maximum-a-posteriori coupling estimation. This approach provides regularization during the early, data-scarce stages of optimization and gradually relinquishes control as task-specific observations accumulate. We instantiate the framework in multitask Bayesian optimization as ICG-MTBO, using directional fitness-class queries to guide inter-task coupling estimation, and further instantiate it in MFEA-II using decision-space-overlap queries to guide random mating probability estimation. Experiments across synthetic benchmarks and a real-world robot arm control problem, together with evaluations under different acquisition functions and evolutionary multitasking, demonstrate the effectiveness and generality of ICG-MTO for few-shot multitask optimization.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

Towards Reliable, Generalizable, and Specific In-Context Knowledge Editing via Multi-Objective Reinforcement Learning

Multi-Objective In-context Knowledge Editing (MO-IKE), a multi-objective RL algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process, enabling more balanced and globally coherent prompt construction.

Xu-Zhong Wang, Maiqi Jiang, Tejal Nair et al. · 1 citation
Conference Open access Sep 2026

Performance-Driven Demonstration Selection for In-Context Learning

Performance-Driven Demonstration Selection (PDDS), which directly aligns demonstration selection with ICL performance, is proposed, which formulates selection as predicting the target LLM’s downstream task performance for a given query–in-context pair, replacing proxy heuristics with a performance-aware objec-tive.

Wen-Qiang Wang, Ming-Bo Yang, Ai-Ping Zhang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Solving Few-Shot Multiobjective Multitask Optimization via Iterative Sequential Transfer

This paper model MTO as a sequence of sequential transfer optimization problems, concentrating evaluations on a single target per iteration, and proposes a likelihood-informed task prioritization mechanism to maximize transfer utility by identifying the task most likely ready for knowledge integration.

Tingyang Wei, Hao-Feng Wu, Ananda Phan Iman et al. · 0 citations

A Task-Level Explanation Framework for Meta-Learning Algorithms

This work introduces TLXML (Task-Level eXplanation of Meta-Learning), a novel framework that extends influence functions to meta-learning settings, enabling task-level explanations of adaptation and inference, and proposes a Gauss-Newton-based approximation.

Yoshihiro Mitsuka, Shadan Golestan, Zahin Sufiyan et al. · 0 citations
#natural language process... Preprint Oct 2026

TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training

This work introduces TrajLong, a novel framework that compiles trajectories into long-context training tasks with dense supervision, targeting three representative atomic capabilities: evidence grounding, cross-evidence aggregation, and temporal state maintenance, and investigates the relationship between long-context...

Miao Peng, Qin-Tong Zhang, Nuo Chen et al. · 0 citations
Book Open access Sep 2026

Attending to the Core: Core-Task Attention for Recommendation

Core Task Attention (CTAtt), a core-task-centric method that employs a novel attention mechanism to adaptively aggregate informative representations from auxiliary tasks into the core task’s prediction tower, and achieves strong performance even when built upon a simple Shared-Bottom architecture.

Jing-Yang Chen, Chen-Ye Sun, Man Zhou et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.