Skip to content
Conference

High-precision image segmentation based on a hybrid CNN–Transformer architecture

Jul 2026 · International Conference on Machine Vision, Automatic Identification and Detection · Vol 14261, pp. 142610Y - 142610Y-6 · 0 citations · 9 references
Engineering

TL;DR

The results indicate that the collaboration between global semantics and local details within a unified weighting domain can effectively improve the separability and deploy ability of high-resolution segmentation.

Abstract

To address the challenge of balancing long-range dependency modeling and detail fidelity in urban scenes, medical slices, and high-resolution remote sensing imagery, this study proposes a lightweight hybrid architecture that integrates a lightweight CNN with a window-based Transformer. In the front-end, depth wise separable convolutions and residual connections are employed to extract edge-sensitive features, complemented by an edge-guided branch and channel recalibration to enhance thin structures and sharp boundaries. In the intermediate stage, multi-scale local window self-attention is utilized to capture long-range dependencies, while learnable windows and sparse global proxies are introduced to reinject global semantic information. In the bridging stage, deformable alignment and gated residual connections are adopted for cross-scale feature fusion, with weights jointly modulated by edge density and global proxies. Experiments are conducted on the Cityscapes validation set under a single-scale inference setting with a resolution of 2048×1024, using an NVIDIA A100 80 GB GPU, batch size of 1, and mixed precision enabled. The proposed method achieves an IoU of 92.3, an F1 score of 93.1, and a speed of 25 FPS, only 12.8 M parameters, 31.6 GFLOPs, and peak GPU memory usage of 2.8 GB. Compared with U-Net (IoU 88.5, F1 89.2, 20 FPS) and SegFormer (IoU 90.1, F1 91.0, 18 FPS), the proposed approach demonstrates clear advantages in accuracy, real-time performance, and intrinsic structural efficiency. The results indicate that the collaboration between global semantics and local details within a unified weighting domain can effectively improve the separability and deploy ability of high-resolution segmentation.

View source

Similar papers

Conference Jul 2026

DenseSwinLight: A Hybrid CNN–Transformer Model with Lightweight Post-hoc Fusion for Visual Explainability

The Hybrid CNN–Transformer architectures offer high-performance image analysis by combining local extraction and global contextual modeling. However, this integration complicates decision-making mechanisms and increases the opacity of models, making them difficult for human users to interpret. We propose DenseSwinLight, a hybrid approach that combines convolution-based feature extraction using DenseNet201 and global contextual modeling using the Swin Transformer V2-Large to simultaneously enhance local discrimination and global context. Beyond performance, the goal of this work is to make the model's decisions more transparent. We introduce a post-hoc explainability module based on the fusion of two complementary sources of evidence, namely a Grad-CAM map from the CNN branch and a proxy attention map. The fusion is learned by an extremely lightweight MLP fusion module, consisting of only 65 parameters, and constrained by area, total variation, and entropy regularizations to produce a parsimonious and stable explanatory mask. Evaluated on a real dataset acquired in an uncontrolled environment covering six classes (Bacteria, Fungi, Healthy, Pest, Phytophthora, and Virus), DenseSwinLight achieves an accuracy of 98.96% on the test set with Test-Time Augmentation, and a macro F1 score of 0.9897. The fusion module achieves a combined soft fidelity score of 0.498 and 0.513 in hard mode according to the insertion and deletion curves, for an average mask area of 0.12. These results confirm the model's ability to provide robust predictions while producing compact and actionable visual explanations for human interpretation.

Deric Claudio Vitasoa, Paul Mahenina Randriamitsiry, Hajarisena Razafimahatratra et al. · 0 citations
Jul 2026

AKNet: an aligned kernel network for image restoration

An efficient aligned kernel network (AKNet) is proposed, which innovatively employs super-large convolution kernels to capture global receptive fields with minimal computational overhead, effectively mimicking the long-range dependency modeling of transformers.

Wan Li, Xiao-Lin Zhang · 0 citations
Open access Aug 2026

GLA-DesnowNet: A Lightweight Hybrid CNN–Transformer Architecture for Image Snow Removal

Single-image snow removal remains a challenging, ill-posed inverse problem in computer vision due to the highly variable appearance of snow degradation. Existing CNN-based methods are limited by local receptive fields and cannot model globally distributed snow patterns, while Transformer-based methods achieve strong performance at a prohibitive computational cost. To address both limitations, GLA-DesnowNet is proposed as a lightweight hybrid architecture that introduces a novel Gated Transformer (GT) module, a gated global attention mechanism designed specifically for physics-driven image desnowing. The Gated Transformer applies global multi-head self-attention with a learned convolutional gate, enabling long-range spatial dependencies to be captured while remaining computationally efficient. An adaptive three-channel snow-mask generation strategy is introduced to automatically produce pixel-level supervision from paired images without manual annotation. A compound loss combining L1 reconstruction with multi-scale pyramid loss is employed to ensure consistent restoration across spatial scales. Evaluated on the Snow100K benchmark, the proposed method achieves 29.30 dB PSNR and 0.93 SSIM in only 160 training epochs, with 0.53 M parameters and 4.44 GFLOPs per inference—significantly fewer than existing state-of-the-art (SOTA) methods—while maintaining competitive restoration quality. On the Comprehensive Snow Dataset (CSD), the identical model achieves a 27.79 dB PSNR and 0.90 SSIM. These results confirm a strong efficiency–accuracy trade-off and cross-dataset generalization suited for resource-constrained and real-time deployment.

Habibulloyev Fakhriddin Abduhalim Ugli, Mst Farjana Aktar, Unal Aras et al. · 0 citations
Open access 2026

A Deformable Hybrid Transformer With Multi-Axis Strip Attention for Geometric-Aware Medical Image Diagnosis

Accurate diagnosis in medical imaging is often hampered by two intrinsic factors: the complex, anomalous geometric distortion of anatomical structures and the extreme size variability of pathological lesions. Existing convolutional neural networks (CNNs) struggle with global context, while Vision Transformer (ViT) models are limited by quadratic computational costs or reliance on window-based attention mechanisms that disrupt semantic continuity. To address these limitations, we propose MaxStripViT, a novel hybrid architecture that effectively integrates distortion modeling with multi-axis strip attention mechanisms and Local Position-Aware blocks. Our method introduces three key contributions: 1) geometric-adaptive stem (GAS) leverages learnable offsets via Deformable Convolutions (DCNv2) to dynamically align the sampling grid with irregular organ boundaries at the earliest feature extraction stage. It effectively mitigates background noise; 2) position-aware local block is proposed to enhance the Mobile Inverted Bottleneck (MBConv) with Coordinate Attention. This mechanism explicitly models long-range dependencies along spatial axes, improving the precise localization of subtle lesions; and 3) novel sequential multi-axis strip attention mechanism is proposed to replace shift window-based self-attention. The proposed method is experimented on three dataset benchmarks of CT and ChestXray imaging for multi-class lung disease such as CheXtImageNet, IQ-OTH/NCCD, and ChestXray-Image dataset. The results demonstrated that MaxStripViT achieves robust performance and outperforms standard ViT models, and improves performance compared to the most advanced hybrid models currently available, including Swin Transformer, MaxViT, CSWin, ConvNeXt, ConvNeXtv2, and EfficientNet-B7, in terms of classification accuracy and computational efficiency. The proposed method provides a robust solution for health scenarios requiring both geometric flexibility and multi-scale interpretability.

Thanh-An Pham, Tuong-Lan Le van, Van-Dung Hoang · 0 citations