Beyond Perceptual Fidelity: Utility and Distortion Modeling for Machine Vision and Embodied Action
Abstract
As visual data are increasingly consumed by recognition models, vision language systems, and robotic agents, perceptual quality alone cannot fully reflect downstream utility. Mild visual distortions may remove boundary, color, semantic, or affordance cues important to machine perception, while stronger perceptual degradation may still preserve enough information for recognition or action. We model machine oriented quality as the task utility retained after visual distortion degrades useful evidence. The model combines signal degradation, structural visibility, semantic consistency, and action critical cues, with score calibration for correlation based evaluation. The same formulation is used for absolute machine vision utility and embodied action utility. Reference relative quality is derived from paired absolute utility estimates of distorted and clean images. We implement the method using frozen visual representations, lightweight regressors, reliability score aggregation, and calibration, without fine tuning large vision language backbones. On the ACM Multimedia 2026 M-VQA Challenge, covering VLM performance prediction, relative degradation assessment, and embodied action utility prediction, the final system achieves official scores of 0.7390, 0.9065, and 0.4739, respectively, and ranks first on Track~2.