Skip to content

A KL Certificate for Best-of-$N$ Reranking in Language-Model Inference

2026 · IEEE Signal Processing Letters · Vol 33, pp. 3307-3311 · 0 citations · 12 references

Abstract

Best-of-$N$ reranking draws independent candidates from a reference policy and selects the response maximal under a fixed, sample-independent strict total order on outcomes. The selected law may differ substantially from the reference in Kullback–Leibler divergence. Prior work introduced a bounded statistic depending only on the accepted response's reference mass and conjectured that its expectation upper-bounds this divergence. This letter proves the conjecture for every finite ordered distribution. The proof applies to the full positive cumulative-distribution-function power family, not only integer sample counts. It combines a strictly monotone binary gauge, a top-atom chain-rule recursion, and induction, and yields an exact nonnegative slack decomposition and a quantitative tightness bound. A beta-quantile representation identifies the universal divergence cap. We also treat reference-preserving reward ties, provide deterministic high-precision illustrations, derive clipped fixed-sample confidence bounds, and specify stable evaluation and exact probability-logging requirements.

View source

Similar papers

Preprint Aug 2026

Ranked spreadness and sample-based testing

In this note, we introduce the notion of ranked spreadness, a strengthening of the usual spread condition in which the elements of each member can be ordered so that their one-coordinate marginals decay geometrically with their rank. This additional structure removes the dependence on the maximum set size in random-containment estimates. We prove width-free hitting and weighted-concentration theorems for ranked-spread set systems, together with an elementary kernel-extraction theorem showing that ranked spreadness arises naturally in arbitrary distributions on small sets. Our main application is to the simulation of nonadaptive property testers by sample-based testers. If a one-sided tester has average query complexity $d$ and rejects every far input with probability at least $\delta$, then, for every integer $c>d/\delta$, it admits a one-sided sample-based simulation with expected sample complexity $O_{d,\delta,|\Sigma|}\bigl(n^{1-1/c}\bigr)$. More generally, if positive inputs are rejected with probability at most $\gamma$ and far inputs with probability at least $\delta>\gamma$, the same conclusion holds for every $c>d/(\delta-\gamma)$. In particular, for constant-query nonadaptive testers we obtain an exponent $1-\Theta(1/q)$, matching, up to the dependence on the rejection gap, the exponent conjectured by Fischer, Lachish, and Vasudev.

Gaia Carenini · 0 citations
Preprint Aug 2026

A Mini-Batch Counterexample to Last-Iterate Convergence in Definable Optimization

We give a counterexample to the convergence conjecture in Remark 12 of [Bolte&Pauwels, 2021] for mini-batch stochastic approximation with definable potentials. The construction uses two convex piecewise-affine, hence semialgebraic, summands on $\mathbb{R}$. We choose a deterministic nonincreasing block stepsize sequence satisfying $\alpha_k = o(1/\log k)$ and an admissible minimum-norm selection from each aggregate batch field. On successive blocks, the iterates form lazy reflected random walks on nested dyadic lattices. An explicit endpoint-cover-time estimate, Markov's inequality, and the first Borel-Cantelli lemma imply that almost surely every sufficiently late block's iterates visit their entire lattice. Consequently, the iterates remain in $[-1,1]$ but do not converge, and their accumulation set is exactly $[-1,1]$, on which the averaged objective is constant. Finally, the construction has $\sum_k \alpha_k^2 =\infty$. Both Chat-GPT 5.6 (Sol) and Gemini Pro 3.1 (DeepThink) were used in the development and drafting of this result.

Weiwei Kong · 0 citations
Preprint Aug 2026

Minimax Quantile Bounds via Information Measures

The results show that sharp converses for minimax quantiles require adapting the information measure to the recovery resolution, whether exact or approximate, and to the tail behaviour of the likelihood ratio.

A. Esposito · 0 citations
Preprint Aug 2026

Dirichlet Follow-the-Leader Closes the Gap in Simultaneous Multiclass U-Calibration

A single, horizon-free algorithm that satisfies the optimal regret rate for every bounded proper loss and also adapt to every smooth proper loss, covering nondifferentiable losses and changes of the active simplex face.

Pahan Dewasurendra · 0 citations
Preprint Jul 2026

Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

Large language model (LLM) cascades reduce inference cost by routing easy queries to a small model and deferring hard queries to a larger one. Production cascades govern this deferral through a confidence threshold, but LLM confidence scores are miscalibrated, the threshold must be tuned per model pair and per domain, and no setting yields a formal bound on cascade accuracy. We introduce \textbf{Conformal Cascade} (CC), a multi-tier inference framework that uses conformal prediction set size as the deferral rule: accept when the calibrated set collapses to a single answer, defer otherwise. The procedure delivers a distribution-free, finite-sample accuracy guarantee. By a per-tier union bound, the prediction set at the accepting tier covers the correct answer with probability at least $1 - K\alpha$ for any user-specified $\alpha$; under a selection-preservation condition (consistent with, but not strictly implied by, our marginal coverage results), the bound tightens to $1 - \alpha$. We further characterise expected cascade cost as an explicit function of $\alpha$ and the calibration-set acceptance rate. Across 18 multiple-choice benchmarks spanning science, medicine, commonsense, and standardized exams, evaluated on two-tier cascades drawn from four open-weight model families, CC strictly improves over the strongest calibration-tuned heuristic cascade on the majority of family--benchmark pairs, with the largest gains on reasoning-heavy benchmarks where majority vote is unreliable; on easier benchmarks the cascade commits the vast majority of queries to the small model at no accuracy cost. Extension to open-ended generation requires an answer-clustering step that we leave for future work. The method requires no model training and only black-box API access.

Yifan Dou, Shikan Lian, Shibo Li · 0 citations

Optimal-Dimension U-Calibration by Bayesian Bootstrap

It is shown that the classical Bayesian bootstrap closes this gap in U-calibration, which asks one online probability fore-caster to have low regret for every bounded proper loss, including losses unknown when the forecasts are made.

Pahan Dewasurendra · 0 citations