Skip to content
Preprint

When Information Is Not Enough: Accuracy-Constrained Thermodynamic Costs of Binary Classification

Sep 2026 · 0 citations · 10 references
Computer Science Mathematics

Abstract

How much entropy must a physical classifier produce to achieve a prescribed accuracy? Rate--distortion theory specifies the minimum information required, but does that information threshold suffice to determine the physical cost? We show that it does not, even for a binary task and a two-state memory. For a uniform binary target observed through a finite symmetric experiment, replacing the classification-error constraint with its necessary mutual-information threshold strictly lowers the infimum of entropy production under a common operation time, integrated mobility budget, and sufficiently large finite transition-rate cap. The separation holds whenever the target error lies strictly between the Bayes error of the observations and chance. Two results establish this physical gap. First, ordering observations by posterior confidence gives exact transport--risk and transport--information frontiers. Observations with the same Bayes error can have different cost frontiers. Second, we construct bounded-rate protocols that realize prescribed encoders with write probabilities below one, starting from exact reset, with an explicit excess cost above the transport bound. An achievable information-constrained cost then falls below a lower bound valid for every task-feasible protocol. Examples with repeated noisy observations illustrate the separation. The results identify a limitation of information-only benchmarks for physical classification: task accuracy and kinetic constraints must be retained explicitly. The cost analyzed is total entropy production during memory writing, excluding data acquisition, controller operation, and subsequent reset.

View source

Similar papers

#machine learning Preprint Sep 2026

How Accurate Is Accurate Enough?

How accurate must a numerical approximation be within a learning system? Primitive error alone cannot answer this question: errors of the same magnitude can have very different consequences for losses, predictions, and gradients at different learning states. We study this question through the learning objective itself....

Ning-Kang Peng, Qian-Feng Yu, Jing Mao et al. · 0 citations
#machine learning Preprint Oct 2026

An Accuracy-Information Tradeoff for Loss-Difference Conditional Mutual Information

Loss-difference conditional mutual information (ld-CMI) uses the smallest of the standard observations in the supersample hierarchy of generalization bounds: it measures what a learner's loss differences reveal about which candidate of each pair it was trained on. Accuracy is known to force information into the model;...

Hazar Yueksel · 0 citations
#machine learning Preprint Sep 2026

ALICE: In-context, Zero-shot, Mutual Information Estimation

For the first time, it is shown that a single model closes the gap with neural estimators trained separately for each distribution, while natively supporting different data dimensionality and sample cardinality, enabling zero-shot MI analysis across scientific domains.

Giulio Franzese, Simone Rossi, Pietro Michiardi · 0 citations
#machine learning Preprint Sep 2026

Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators

The FeeBee framework is extended, originally proposed for evaluating Bayes-error estimators, to the optimal BER and AUC, and the resulting procedure provides practical evaluation scores without requiring knowledge of the optimum.

Ryota Ushio, Takashi Ishida, Masashi Sugiyama · 0 citations
#machine learning Preprint Sep 2026

The cost of useful natural gradient updates

What information is needed to turn a natural-gradient direction into a useful finite update? Under a population Kullback-Leibler (KL) budget, we call a step useful if it is feasible and loses at most a fraction $\varepsilon$ of the best feasible gain along the direction. We construct a four-state exponential family who...

Subhransu S. Bhattacharjee, Dylan Campbell, Rahul Shome · 0 citations
#machine learning Preprint Sep 2026

When Does Low-Bit Quantization Preserve the Decisions of Vector Search?

A distribution-free decomposition: the probability that a comparison flips is bounded by the probability mass of exact margins near zero plus the tail probability of the calibrated residual, and a deterministic coupling theorem for Vamana neighbour selection is proved.

Wen-Xuan Xiao, Xu Cao · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.