May 2026· arXiv.org· Vol abs/2605.07304· 0 citations· 41 references
Computer Science
TL;DR
A posterior-sampling algorithm is proposed and shown that both are competitive with full-prior latent bandits when same-state instances share reward parameters, and preferable to them when reward scales differ between instances with the same latent state.
Abstract
Bandit algorithms solve diverse sequential decision-making problems, but are often too sample-inefficient for from-scratch personalization. To substantially reduce exploration times, latent bandit algorithms exploit cross-instance structure implied by discrete latent states, provided that the posterior distribution of rewards and latent states is known and accurate. However, obtaining an accurate model of this structure is difficult, and a small number of latent states may be insufficient to characterize the reward distributions in all problem instances. We propose latent order bandits (LOB), relaxing the assumptions of latent bandits to require only prior knowledge of a \emph{partial order} of action preferences in each state. This allows instances of the same state to vary in reward distributions, as long as the partial order of actions is shared. For example, groups of users on a streaming service may agree on which movie genres are the best but rate experiences on different scales. We give an upper-confidence bound procedure for the LOB problem, applicable to both total and partial latent orders, and give an upper bound on its regret. To improve empirical performance, we propose a posterior-sampling algorithm and show, in a suite of experiments, that both are competitive with full-prior latent bandits when same-state instances share reward parameters, and preferable to them when reward scales differ between instances with the same latent state.
The stochastic linear bandit, where actions are represented as vectors and rewards are linear, is a central paradigm for sequential decision making. We study a partially observed variant of this problem in which the learning agent only sees a random subset of coordinates for each action. Such partial observability arises naturally in settings like recommendation and healthcare, where full action descriptions can be expensive or even impossible to obtain. In general, this makes sublinear regret information-theoretically impossible. However, we show that this barrier can be overcome when the action vectors have low intrinsic dimension. We propose an algorithm, TOFU-POV, that estimates the latent action subspace using the masked actions, imputes current actions using an epoch-wise frozen representation, and runs OFUL in the resulting low-dimensional coordinates. Our theory shows that TOFU-POV enjoys a $\sqrt{T}$ regret that scales with the intrinsic action subspace dimension as opposed to the ambient dimension and quantifies the interaction between these quantities and the missingness, decision set size, and subspace conditioning. We also devise a rank-adaptive algorithm that does not require the knowledge of the intrinsic dimension. We complement these guarantees with a lower bound based on a novel product construction that separates usual reward-learning uncertainty from a missingness-dependent cost intrinsic to partial observation. Synthetic and real data experiments support our theory and show that TOFU-POV can substantially improve upon natural baselines in this challenging problem.
Gautam Dasarathy, V. Gattani, Lalit Jain· 0 citations
Consider a scenario where a decision-maker faces a row of slot machines, each offering unknown and varying payout rates. The objective is to maximize cumulative rewards, yet each action simultaneously provides new information. This problem—balancing the exploration of new options against the exploitation of known rewarding ones—is termed the multi-armed bandit problem. It has evolved from a simple gambling question into a fundamental tool for modern computer systems. This paper looks at three main ways to solve this problem: Explore-Then-Commit, Upper Confidence Bound, and Thompson Sampling. Through careful testing, Thompson Sampling stands out as the best performer, cutting total regret down to 0.60 where UCB reaches 3.63. It also handles delays well—when feedback comes 1000 steps late, Thompson Sampling's lead over UCB grows to nearly 4 times. The paper shows where these methods are used in real life: online ads, where they improve click rates by about 11%; movie and music suggestions, where they help new users find content they like; and medical trials, where they can put over 80% of patients on better treatments. The paper ends with a look at new research areas, including methods that adapt to changing conditions and handle complex choices.
Ziyue Zhou· Applied and Computational En...· 0 citations
This paper uncovers general properties of optimal information structures by exploiting a linear-programming formulation of information design. A critical observation is that an optimum can be found as ``sparse,''i.e., many coordinates of the action-state joint distribution are zero. This implies that, once part of an action-state profile is fixed, there is limited room for the remaining part to fluctuate. As a result, agents'action recommendations are conditionally deterministic in many states, or correlated in a way that allows some agents to infer others'recommendations. The implications of sparsity are illustrated in an adoption problem, where the designer maximizes the number of adopters of an innovation that features network effects. The optimal information structure deterministically recommends full adoption in high states, while it randomizes over nested action profiles in low states, so that whenever an agent is recommended to adopt, she is certain that more optimistic agents also adopt.
Stochastic-process models are, as a rule, far easier to simulate than to condition. Non-linear observations, non-Gaussian likelihoods, black-box information, and global constraints all induce intractable conditional laws, requiring bespoke, model-specific constructions. We introduce LatentFlow, a single framework for conditioning stochastic processes, with no learned neural approximations and no training. Our starting point is to write the stochastic process as the deterministic image of a tractable latent innovation, $f_0 = T_{\vartheta}(\xi_0)$, with $\xi_0$ sampled from a simple reference distribution. This reduces process-level conditioning to latent-space inference: pull the likelihood back through $T_{\vartheta}$, sample the resulting latent law with a tractable guided probability flow, and push the samples forward. This construction is provably exact at the level of the target law; in practice, approximation enters only through finite terminal noising, Monte Carlo guidance, and time discretisation of the continuous-time dynamics, each of which is explicit and systematically reducible. As LatentFlow is training-free, conditioning reduces to solving a single reverse-time SDE. This enables conditional sampling in seconds on a single desktop CPU across model classes that have never shared a scalable method: classical spatial priors, nonlinear stochastic dynamics, mechanistic models from the physical and life sciences, stochastic PDEs, heavy-tails and extremes, point and discrete-state processes, and neural or simulator-defined processes.
Louis Sharrock, L. Astfalck, Henry Moss· 0 citations
Heavy-tailed distributions arise naturally in sequential decision-making problems such as financial investment, online advertising, and network management, where rare but extreme outcomes can dominate performance. Heavy-tailed bandits model online decision-making in these settings by assuming only that rewards $X$ satisfy $\mathbb{E}[|X|^{1+\epsilon}]\leq u$, for some tail exponent $\epsilon\in(0,1]$ and moment bound $u<+\infty$. However, most existing regret minimization algorithms require these parameters to be known. This assumption is particularly restrictive in practice: $\epsilon$ and $u$ govern the frequency and magnitude of rare events and are therefore precisely the quantities that are hardest to infer reliably from limited observations. Motivated by an open problem posed by Genalti and Metelli at COLT 2025, we resolve the assumption-free adaptation problem for heavy-tailed bandits and characterize the price in the regret of not knowing the tail parameters. We first study adaptation to the moment bound $u$ for a fixed tail exponent $\epsilon$. We prove that every algorithm unaware of $u$, or of any upper bound on it, must obey a sharp trade-off between its distribution-dependent and distribution-free regret guarantees. We then introduce a scheduled-exploration algorithm that requires no knowledge of $u$ and matches the resulting adaptation frontier up to logarithmic factors. Finally, we show that the same algorithm can be instanced without knowing $\epsilon$ by calibrating its exploration schedule to the endpoint $\epsilon=1$. It achieves sublinear regret for every fixed $\epsilon>0$, while no algorithm can guarantee sublinear regret uniformly over all $\epsilon\in(0,1]$. Altogether, our results resolve the COLT open problem without additional distributional assumptions and provide a sharp characterization of the statistical cost of adapting to unknown heavy tails.
We consider Bayesian bandit models and prove that Thompson sampling makes at most twice the expected number of mistakes (selections of a suboptimal arm) as any other policy. Our analysis applies as long as the latent arm processes are independent and each arm evolves only when played. For stochastic bandits with best arm defined via mean reward, this confirms a conjecture of Guha and Munagala from 2014, where the factor $2$ is already best possible. The result holds under any nonincreasing sequence of round weights, including fixed horizon and geometric discounting.
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
