Generalized least squares (GLS) analysis shows that, under exchangeability, the optimal symmetric linear combiner of latent embeddings is uniform, supporting majority vote as the natural default in standard SC while leaving room for weighting or pruning under heterogeneous prompt-template branches.
Abstract
Multi-path reasoning methods such as self-consistency (SC) sample $K$ reasoning paths and choose the most frequent answer. However, their gains quickly plateau as $K$ increases, and locating this saturation point currently requires fitting a scaling model to accuracies measured at several values of $K$ or stopping online per query. We formalize multi-path LLM reasoning as a diversity combining problem from wireless communications: each path is a noisy channel observation, and the pairwise correlation of path correctness caps the design-effect effective sample size of the vote at a finite ceiling. Generalized least squares (GLS) analysis shows that, for paths with equal gains, the optimal linear combiner of latent embeddings is uniform exactly when every row of the path covariance has the same sum, as under exchangeability, supporting majority vote as the natural default in standard SC while leaving room for weighting or pruning under heterogeneous prompt-template branches. Across 5 models and 12 benchmarks, prompt-template diversity reduces path correlation in $55$ of $60$ model-benchmark cells, with a mean relative reduction of $16\%$ and reductions in every task domain. We derive an Adaptive-K rule that uses a four-path pilot to select $K^*$, retaining $96$--$103\%$ of MV@$K{=}32$ accuracy across Math, QA, and NLU.
Experiments across diverse reasoning tasks, including MATH500, HumanEval, and GPQA, show that APS consistently outperforms power sampling with a fixed sharpening exponent, without additional training.
Bing-Nan Xiao, Chen-Hao Yang, Bing-Cong Li et al.· 0 citations
Flexible Self-Consistency (FSC) is proposed, which estimates problem difficulty as a continuous signal and dynamically adjusts the number of generated reasoning paths accordingly and leverages it as an indicator of model uncertainty to flexibly control the sampling budget.
Sihyeong Yeom, Geon Park, Geunyeong Jeong et al.· 0 citations
The results suggest that the distribution of additional reasoning paths generated by the algorithm is heavy-tailed and that substantial capacity to overcharge remains even under the best possible audit designed to keep the false-positive rate below $\alpha = 0.1$.
Tori Qiu, A. Velasco, M. Gomez-Rodriguez· arXiv.org· 0 citations
Verified solutions are not equally useful for preparing reasoning models for reinforcement learning (RL). We present a comprehensive study of route diversity, the variation in the sequences of reasoning steps in supervised fine-tuning (SFT) data, and propose a lightweight, rule-based fingerprint to select for it. From...
Chopthin-Consensus Power Sampling (CCPS) is introduced, demonstrating that diversity-preserving resampling and diversity-aware selection are complementary mechanisms for training-free LLM reasoning.
Differentiable Hierarchical Fusion is presented, a novel framework that merges reasoning models with efficient base models via differentiable optimization to produce concise, accurate outputs and introduces a dual-factor adaptive weighting mechanism to capture intra-block variance and inter-block importance hierarchies...
Chuan-Gen Gao, Wen-Lun Zhang, Shang-Kun Wang et al.· Proceedings of the Thirty-Fi...· 0 citations