Results show that BLV-focused post-training improves both accessibility-specific spatial grounding and general visual-text reasoning, and is substantially outperforms the baseline across various benchmarks.
Abstract
An estimated 1 billion people worldwide live with vision impairment, yet current vision-language models (VLMs) produce descriptions too vague for safe navigation by blind and low-vision (BLV) users. Large VLMs can generate high-quality audio-description-compliant narrations but cannot run on mobile devices; small VLMs offer competitive latency but lack spatial detail, directional cues, and hazard awareness for navigational assistance. We present Smol-VL-BLV, a compact VLM for blind and low-vision users that closes this gap using a 500M decoder transformer model and two post-training mechanisms: (1) teacher-student distillation and (2) Group Relative Policy Optimization (GRPO) with a composite BLV reward targeting directional language, metric distances, and hazard detection. Because multi-stage post-training can induce catastrophic forgetting, we add a lightweight finetuning stage after the last stage GRPO finetuning to recover general descriptive quality while preserving BLV-specific spatial grounding. Our best model substantially outperforms the baseline across various benchmarks, including tasks: VQA, BLV captioning, OCR, and latency. Compared with the baseline for relative improvement, it improves the Spatial score gain of 19.3%, and the Social score gain of 14.8%. It also increases OCR-Bench by 101.5%, and raises TextVQA accuracy by 44.2%. These results show that BLV-focused post-training improves both accessibility-specific spatial grounding and general visual-text reasoning. Deployed on a mid-range Android smartphone via Mixed-Precision Quantization, the model remains approx. 450 MB and runs entirely on-device, offline and without network dependency, generating descriptions with latency dependent on host hardware capabilities. Our model, dataset, and code is publicly released at https://smol-vl-blv.github.io/Smol-VL-BLV-website/
A novel end-to-end framework that integrates LLMs, VLMs and digital twin technologies to deliver a spatially cognitive navigation support for visually impaired and neuro-divergent users is introduced.
Hamza Riaz, Jaime B. Fernandez, Ian Mills et al.· 0 citations
Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...
Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al.· European Conference on Compu...· 0 citations
This paper presents the first comprehensive survey of VLM applications in low-level vision, proposing a unified taxonomy that organizes existing literature into two complementary paradigms, and identifies critical open challenges, including semantic-pixel misalignment and computational efficiency.
Feng-Yang Xiao, May May Liqeliqe Mar. Liqe Mar., Mariusz Dec et al.· 0 citations
Vision-language-action (VLA) models encode semantic information from vision-language pretraining, but manipulation also requires precise spatial reasoning. We present GeoBridge-VLA, a two-stage method for learning geometric features from a pretrained VLA's frozen visual encoder and using them for action prediction. Sta...
Hyunjung Song, Kangmin Kim, Loren Jinsoo Um et al.· 0 citations
O2C-Nav is proposed, an efficient zero-shot navigation framework that calls only a single large model once per decision step and introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers...
Shi-Qi Pan, Qi Zheng, Hanmeng Sun et al.· 2 citations· ⚡1
VisLens (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head, and which matches or exceeds prior baselines while delivering a substantial latency advantage.
Jingfeng He, Sanghwan Kim, Zeynep Akata· 0 citations
The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.