Skip to content

Small yet Assistive: Spatially-Aware Post-Training for Low Vision

Sep 2026 · 0 citations · 24 references
Computer Science

TL;DR

Results show that BLV-focused post-training improves both accessibility-specific spatial grounding and general visual-text reasoning, and is substantially outperforms the baseline across various benchmarks.

Abstract

An estimated 1 billion people worldwide live with vision impairment, yet current vision-language models (VLMs) produce descriptions too vague for safe navigation by blind and low-vision (BLV) users. Large VLMs can generate high-quality audio-description-compliant narrations but cannot run on mobile devices; small VLMs offer competitive latency but lack spatial detail, directional cues, and hazard awareness for navigational assistance. We present Smol-VL-BLV, a compact VLM for blind and low-vision users that closes this gap using a 500M decoder transformer model and two post-training mechanisms: (1) teacher-student distillation and (2) Group Relative Policy Optimization (GRPO) with a composite BLV reward targeting directional language, metric distances, and hazard detection. Because multi-stage post-training can induce catastrophic forgetting, we add a lightweight finetuning stage after the last stage GRPO finetuning to recover general descriptive quality while preserving BLV-specific spatial grounding. Our best model substantially outperforms the baseline across various benchmarks, including tasks: VQA, BLV captioning, OCR, and latency. Compared with the baseline for relative improvement, it improves the Spatial score gain of 19.3%, and the Social score gain of 14.8%. It also increases OCR-Bench by 101.5%, and raises TextVQA accuracy by 44.2%. These results show that BLV-focused post-training improves both accessibility-specific spatial grounding and general visual-text reasoning. Deployed on a mid-range Android smartphone via Mixed-Precision Quantization, the model remains approx. 450 MB and runs entirely on-device, offline and without network dependency, generating descriptions with latency dependent on host hardware capabilities. Our model, dataset, and code is publicly released at https://smol-vl-blv.github.io/Smol-VL-BLV-website/

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Assisted Spatial Cognition Through Vision-Language Models

A novel end-to-end framework that integrates LLMs, VLMs and digital twin technologies to deliver a spatially cognitive navigation support for visually impaired and neuro-divergent users is introduced.

Hamza Riaz, Jaime B. Fernandez, Ian Mills et al. · 0 citations
Conference Open access Sep 2026

Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

Generative vision-language models (VLMs) such as Qwen-VL and LLaVA achieve strong zero-shot performance on tasks overlapping with their pretraining distribution, yet fail on specialized domains where the required discriminative features were never learned, a regime we term distant out-of-distribution (OOD). Standard ad...

Hung-Jen Chen, Yuek F. Ho, Ting-Yao Huang et al. · 0 citations
Review

Toward Multimodal Understanding in Low-Level Vision: The Rise of Vision-Language Models

This paper presents the first comprehensive survey of VLM applications in low-level vision, proposing a unified taxonomy that organizes existing literature into two complementary paradigms, and identifies critical open challenges, including semantic-pixel misalignment and computational efficiency.

Feng-Yang Xiao, May May Liqeliqe Mar. Liqe Mar., Mariusz Dec et al. · 0 citations
Preprint Oct 2026

GeoBridge-VLA: Geometry-Aware Residual Adaptation for Vision-Language-Action Models

Vision-language-action (VLA) models encode semantic information from vision-language pretraining, but manipulation also requires precise spatial reasoning. We present GeoBridge-VLA, a two-stage method for learning geometric features from a pretrained VLA's frozen visual encoder and using them for action prediction. Sta...

Hyunjung Song, Kangmin Kim, Loren Jinsoo Um et al. · 0 citations
#artificial intelligence Preprint Sep 2026

One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints

O2C-Nav is proposed, an efficient zero-shot navigation framework that calls only a single large model once per decision step and introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers...

Shi-Qi Pan, Qi Zheng, Hanmeng Sun et al. · 2 citations · ⚡1
#small language model Preprint Aug 2026

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

VisLens (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head, and which matches or exceeds prior baselines while delivering a substantial latency advantage.

Jingfeng He, Sanghwan Kim, Zeynep Akata · 0 citations

Related blog posts

MIT News · Artificial Intelligence Jun 3, 2026

MIT researchers teach AI models to interpret charts

The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.