It is suggested that human gaze provides an effective and practical guidance signal for improving the computational efficiency of Vision-Language Models without requiring changes to the underlying models, offering a simple and general approach for reducing visual inference cost while largely preserving answer quality.
Vision-language-action (VLA) policies must ground language instructions in the right objects, contact points, and target locations for reliable manipulation. Human gaze provides a natural signal for this grounding, but existing robot datasets rarely pair real human gaze with the multi-camera observations used by modern...
Anupam Pani, Wang-Gui Yuan, Qi-Fan Wang et al.· 0 citations
Predicting human gaze is a core capability for applications ranging from web/UI design analysis to robotics and human-computer interaction. Yet, most visual attention modeling methods output only a dense saliency map, which is often insufficient for action: practitioners need to connect attention peaks to discrete elem...
eye-tracker-supervised gaze prompting is introduced, which uses gaze recorded during VR teleoperation to provide frame-level visual guidance for VLA fine-tuning.
Yi-Han Zhou, Rui Yan, Ming-Cong Li et al.· arXiv.org· 0 citations
Visual perception should be iterative and task-directed, with representations refined in response to the question and the need for further processing. While Vision-Language Models (VLMs) increasingly support extended reasoning, their visual tokens are still typically acquired in a single, task-agnostic encoder pass. Th...
Wen Ye, An-Ning Hu, Hong-Ye Fang et al.· 0 citations
Results show that pretrained convolutional representations, when integrated through learned adaptation and fusion, can improve the visual understanding of existing VLMs without replacing their original visual encoders.
Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine-grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as...
Ze-Rui Chen, Chang-Rui Chen, Fei Ni et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.