This work considers discrete distributions with hidden forest structure, accessed through a fixed approximate conditional oracle, and studies whether total evaluations, including discovery, can be sublinear in sequence length $N; sublinear sequential depth then follows.
KME-CLIP is proposed, which leverages this structure through the inner product in a reproducing kernel Hilbert space (RKHS) and theoretically proves that the KME-CLIP similarity can bring the contrastive loss arbitrarily close to its optimal value, which is attained by PMI, as the size of the point set grows.
This paper theoretically analyzes the MaskGIT sampler for image modeling, revealing its implicit temperature sampling mechanism, and shows that MaskGIT is asymptotically equivalent to a choose-then-sample (CTS) formulation, instantiated as the "moment sampler," which explicitly separates index selection from token samp...
It is shown that the population-optimal UDM predictor generally depends on time: time controls how much the model should trust the observed context, and this dependence can become negligible in finite-data settings relevant to language.
Chunsan Hong, Chieh-Hsin Lai, Satoshi Hayakawa et al.· 0 citations
Uniform-state discrete diffusion models update all tokens in parallel while keeping every position revisable. Even when the commonly used top-$p$ rule leaves only one candidate at a position, that choice affects only the current reverse step and can be revised at the next sampling step. We ask what changes when selecte...
Simplax is introduced, an exact Dirichlet--categorical augmentation that couples each corrupted categorical state with an auxiliary simplex-valued variable while preserving the original uniform diffusion process as its categorical marginal.
Jinya Sakurai, Patrick Pynadath, Satoshi Hayakawa et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.