Skip to content
Conference Open access

Cross-attention multimodal transformer: adaptive segmentation of urban functional areas by fusing street view and satellite imagery

Sep 2026 · International Conference on Image Processing and Pattern Recognition · Vol 14388, pp. 143880F - 143880F-7 · 0 citations · 10 references
Engineering

TL;DR

A cross-attention multimodal Transformer is proposed that integrates street-view cues, such as building facades and business types, with satellite features, including roof shape, block texture and road structure to address the deficiency of single-source imagery.

Abstract

Urban functional area division helps to improve urban planning, new-area construction and public service distribution. To address the deficiency of single-source imagery, this paper proposes a cross-attention multimodal Transformer that integrates street-view cues, such as building facades and business types, with satellite features, including roof shape, block texture and road structure. Quality scoring, geometric alignment, cross-modal attention and uncertainty gating are used to adaptively segment the six functional areas of an urban area, reaching an mIoU of 88.6% and an F1 score of 90.1%, and surpassing SegFormer-B0. The way is still relatively robust in low light and under occlusion or location perturbation to provide some visual evidence for urban planning.

Read PDF

Similar papers

Conference Sep 2026

Road-Scene Visibility Estimation Based on ResNet-18 Dual-Branch Enhanced Gated Fusion

Road-scene visibility estimation is a critical task in intelligent transportation systems and autonomous driving perception. To address the spatially non-uniform degradation present in foggy images and the difficulty of single-scale features in simultaneously characterizing global atmospheric conditions and local objec...

Xue-Jun Wang, Jin-Jian Xiao, Yun-Tao Ma et al. · 0 citations
Open access Aug 2026

CLIP-RSICD: A Fine-Tuned Vision Transformer for Urban Land Cover Classification

Reliable mapping of intra-urban land-cover categories at fine spatial and thematic resolutions is fundamental to evidence-based spatial planning. Nevertheless, transferring classification models across territories remains difficult because urban morphology varies regionally. This challenge is pronounced in South Americ...

F. J. da Costa, R. Pescim, M. R. Urbano · 0 citations
Open access Sep 2026

Multi-Target Remote Sensing Segmentation with Cross-Scale Attention for Urban-Ecological Intelligence

Remote sensing image segmentation is essential to extract valuable information from satellite and aerial images to achieve significant applications such as urban planning and ecological monitoring. Yet, it is hard to accurately segment diverse and complicated features because of the constraints of conventional appro...

Chibueze Favour Aririguzo · 0 citations
Preprint Sep 2026

Instance Segmentation and Fine-grained Classification for Urban Buildings with Adaptive Region Dividing and Spatially-Supervised Contrastive Learning

Accurate instance-level and functional understanding of urban buildings in large-scale point clouds is essential for digital city modeling and urban analysis. However, the extensive spatial coverage of urban scenes leads most existing methods to rely on predefined blocks for training and evaluation, although such parti...

Wei-Yuan Zhang, Qi Zhang, Hui-Dan Huang · 0 citations
Open access Jul 2026

Pyramid attention network for indoor scene segmentation

With the rapid development of artificial intelligence technology, indoor scene segmentation is more and more widely used in robot vision, security monitoring, firefighting, and other fields. Usually, indoor objects have problems such as easy mutual occlusion and dense layout, thus requiring high accuracy of network seg...

Rui Cao, Xiang Chen, Yi-Qin Wang et al. · 0 citations
Open access Aug 2026

BRPE-UNet: A Multi-Scale Feature Enhancement Network with Polarized Self-Attention and Adaptive Edge Refinement for Farmland Road Extraction

The precise extraction of field roads in hilly and terrace farmlands poses great challenges for remote sensing analysis. Farm roads are narrow and irregular, with spectra close to crop ridges, and frequently covered by tree and building shadows. Traditional segmentation methods easily produce missed detection, broken a...

Shuai Yang, Xiao-Yan Meng, Liang Yu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.