2026· Proceedings of the 4th International Conference on Image, Algorithms, and Artificial Intelligence, ICIAAI 2026, 22-24 May 2026, Singapore, Singapore· 0 citations· 8 references
TL;DR
It is suggested that standard strategies need structural changes, like subsampling, to work efficiently in large-scale environments to handle a massive number of sub-optimal arms.
Abstract
. This study tests how Explore-Then-Commit (ETC), Upper Confidence Bound (UCB), and Thompson Sampling (TS) behave in a static setting where the action space size, K, varies from 10 to 2,400. A Fixed-Gap Gaussian Environment based on MovieLens data is used to ensure the task difficulty stays the same. Experiments show that performance gets worse for every algorithm as K increases. ETC fails the most, with identification accuracy dropping below 20% because it wastes too much budget on exploration. UCB and TS do better but still show linear regret growth. This implies that probabilistic exploration is not enough to handle a massive number of sub-optimal arms. The findings suggest that standard strategies need structural changes, like subsampling, to work efficiently in large-scale environments.
. Multi-armed bandit (MAB) algorithms are commonly used in sequential decision tasks such as online recommendation and advertising. In real-world systems, algorithms often have limited time and data to learn from, and poor decisions can be costly. From a finite-horizon perspective, this paper presents a mechanism-level...
Yi-Chen Fan· Proceedings of the 4th Inter...· 0 citations
Existing cumulative regret guarantees for Bayesian optimization (BO) do not necessarily characterize the quality of the algorithm's final output. This gap is critical under noisy evaluations, where the goal is to return a single high-quality recommendation. To address it, we introduce efficient exploration strategies t...
Hwanwoo Kim, Jungwoo Hong, Kyungjae Lee· 0 citations
The results provide a modular framework that can be used to analyze linear bandit algorithms and explicitly connect quality of parameter estimation to optimal regret accumulation.
Arda Güçlü, Subhonmesh Bose, J. Birge· 0 citations
We study fixed-budget adaptive experimentation aimed at maximizing the probability of obtaining sufficient posterior evidence that a recommended design's mean performance exceeds a target. Unlike best-arm identification, this objective does not require finding the best design. For Gaussian bandits with a Gaussian prior...
Yi-Nan Li, Kwang-Sung Jun, Kapilan Balagopalan et al.· 0 citations
We study stochastic contextual bandits with an unknown context-lumpable structure. The $S$ contexts are partitioned into $r$ blocks, and every arm has the same mean reward across contexts in the same block. Under arbitrary context distributions and without separation assumptions, we design an algorithm with expected re...
Feel-good Thompson sampling (FGTS) augments the posterior of vanilla Thompson sampling with an exploration term that favors reward functions with high predicted optimal values. This modification achieves minimax-optimal regret up to logarithmic factors in linear contextual bandits, but existing analyses of its two stan...
