Entropy functionals and their associated divergences underlie many statistical methods, including maximum entropy inference, minimum divergence estimation, and goodness-of-fit testing, yet choosing among Shannon, R\'enyi, Tsallis, and more general entropies is often a matter of convention rather than structural principle. We introduce a measure theoretic framework in which admissibility requires the entropy of an input measure to be bounded above by that of its reference measure whenever the former is absolutely continuous with respect to the latter. Under generalized mean-value composition, we characterize all such entropy functionals and obtain a four-level hierarchy determined successively by the mean generator, entropy scale, and additivity assumptions. A continuous strictly monotone generator $g$ is admissible exactly when $t\mapsto g(1/t)$ is strictly convex for increasing $g$, or strictly concave for decreasing $g$. This resolves a question posed by R\'enyi (Proc. 4th Berkeley Sympos. Math. Statist. Prob., 1961) concerning which generalized means may replace the arithmetic mean in his entropy axiomatization. The same criterion is equivalent to strict convexity of an associated Csisz\'ar $f$-divergence generator and therefore yields data processing under Markov kernels with an exact equality condition. Within this hierarchy, product additivity singles out the R\'enyi family, while internal additivity, or product additivity together with arithmetic mean-value composition, singles out Shannon entropy. The characterization is constructive and yields new admissible entropy and divergence families, including integral-transform examples.
Maximum-entropy reference distributions are usually constructed on the normalized probability simplex. This formulation is less natural for unnormalized statistical models, in which positive multiples represent the same shape, and it does not directly explain how a prescribed admissible region should determine the deformation parameter of a bounded-support reference distribution. We formulate maximum entropy on the projective space of nonnegative measures and establish three results of statistical relevance. First, a universality theorem shows that every admissible monotone transform of the same normalized power functional has exactly the same optimizer under linear moment constraints. The result unifies the maximum-entropy implications of Tsallis and R\'enyi entropies, H\"older composite scores, pseudo-spherical scores, Bregman--H\"older constructions, and related homogeneous divergences without asserting a new distribution family. Second, the common optimizer is characterized as a $q$-exponential density; under mean and covariance constraints it is a compactly supported $q$-Gaussian for positive deformation and a Student-type density for negative deformation. Third, a prescribed Mahalanobis acceptance region with squared radius $R^2>d+2$ uniquely determines the deformation parameter $\gamma_R=2/(R^2-d-2)$. The resulting affine-equivariant reference density is the unique projective maximum-entropy solution, and its support coincides with the specified ellipsoid without an additional support constraint. This provides a principled method for constructing bounded-support statistical reference distributions from robust location and scatter estimates or from externally specified admissible regions.
Let $X,X'$ be independent and identically distributed discrete real-valued random variables of finite Shannon entropy, and write $H(X)$ for the Shannon entropy of $X$. We prove that \[ \max\{H(X+X'),\,H(XX')\} \ge \frac87 H(X)-O(\log H(X)). \] This is the entropic analog of the celebrated sum-product phenomenon, and answers a question of Goh, which simply asked for a coefficient strictly larger than 1. An example by the author, Gavalakis, and Kontoyiannis showed the coefficient cannot exceed $\frac43$. Previous work by Gavalakis, Goh, and Kontoyiannis was able to prove a result of a weaker form, which could not translate to a coefficient strictly larger than 1 because of examples where the min-entropy is significantly smaller than the Shannon entropy. By splitting the distribution of $X$ into uniform pieces, which costs $O(\log H(X))$ entropy, we obviate this issue, establishing a coefficient of $\frac{10}{9}$. We augment this to $\frac87$ by adapting the work of Solymosi, which established the combinatorial sum-product phenomenon with coefficient $\frac43$ by bounding the multiplicative energy, to the entropy setting, again via a uniformization technique.
We propose a geometric--analytic framework for equilibrium statistical mechanics on infinite-dimensional Hamiltonian systems. In situations where no suitable $\sigma$-additive invariant measure is available, we use \emph{normalized means}, which generalize probability measures and normalized integrals. This construction yields entropy and free-energy functionals on weak symplectic Fr\'echet manifolds and gives existence and uniqueness of exponential-family equilibrium states under explicit admissibility and separation assumptions. These states are stationary under Hamiltonian flows preserving both the reference mean and the equilibrium weight, and satisfy a classical Poisson--KMS identity when the reference mean is Poisson invariant. Under a local exponential regularity assumption, the logarithmic partition functional is smooth and convex, with Hessian given by the covariance form. It is strictly convex modulo thermodynamically null directions and, through Legendre--Fenchel duality, induces a concave entropy on the domain of extensive variables. We illustrate the framework with $H^s$-geodesic equations on current groups $\operatorname{Map}(M,G)$ and diffeomorphism groups $\operatorname{Diff}(M)$, including hydrodynamic and field-theoretic examples.
In this paper, we derive a conditional copula representation for expectations of the form $\mathbb{E}[g(\boldsymbol{X})]$, where $\boldsymbol{X}$ is a random vector with arbitrary marginal distributions and $g$ is a measurable function satisfying suitable integrability conditions. The proposed representation explicitly separates the contributions of the marginal distributions and the dependence structure through conditional copula distributions, yielding a unified quantile--copula framework for a broad class of statistical functionals. This framework encompasses numerous quantities of practical interest, including moments, probabilities, dependence measures, inequality indices, entropy measures, and multivariate functionals. We further establish extremal bounds under fixed marginals by exploiting the concordance order on copulas and characterize the classes of functions for which these bounds apply through the notion of $\Delta$-antitonicity. Finally, several illustrative examples illustrate the versatility of the proposed framework through applications to risk measures, stochastic superiority probabilities, information measures, and option pricing under dependence uncertainty.
R. Vila, C. Otiniano, Carolyne Brito et al.· 0 citations
It is shown that the MEM dual problem admits a reformulation as an expected risk minimization problem, thereby placing MEM within the modern framework of stochastic optimization and enabling scalable stochastic gradient algorithms for large-scale inverse problems.
Matthew King-Roskamp, Gabriel Rioux, R. Choksi et al.· 0 citations
Several new and sharp informational inequalities are derived as a byproduct of Stam-like and moment-entropy-like inequalities in the relative framework and a recently established inequality mixing the R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross entropy of suitable probability density functions. More precisely, we obtain a Stam-like inequality connecting the R\'enyi entropy power, the recently introduced scaling-invariant relative Fisher information and the R\'enyi cross entropy. Furthermore, we derive an inequality involving only Fisher-like informational measures and another inequality involving only moment-like functionals of non-relative, relative and cross types, respectively. All the inequalities are sharp. The minimizers of the Stam-like inequality are, in certain cases, pairs of Gaussian or stretched Gaussian probability densities; in contrast, each minimizer of the moment-like inequality is the probability density of the generalized Beta distribution.