Oct 2026· Journal of Transportation Engineering Part A Systems· Vol 152· 0 citations· 23 references
TL;DR
A visual foundation model-based multilabel perception framework that leverages existing on-board surveillance videos without requiring additional sensors or manual annotation is proposed, enabling zero-shot recognition of diverse environmental elements.
Abstract
Reliable perception of the train operating environment is essential for supporting efficient and intelligent railway operations. However, traditional trackside sensing infrastructures are sparsely deployed and costly to maintain, making it difficult to obtain whole-process environmental information. To address this challenge, this study proposes a visual foundation model-based multilabel perception framework that leverages existing on-board surveillance videos without requiring additional sensors or manual annotation. The framework utilizes a visual foundation model to generate initial open-vocabulary semantic tags, enabling zero-shot recognition of diverse environmental elements. A semantic refinement mechanism is then introduced to extract controllable environmental labels through similarity matching with a predefined label library. Finally, a dynamic label correction module integrates prior knowledge and temporal cues to suppress frame-level noise and ensure sequence-level consistency. Experiments on a real-world on-board video dataset demonstrate the effectiveness of the proposed framework, achieving an image-level label accuracy of 91.8% and an event-level perception accuracy of 90.6%. This work provides a practical and generalizable pipeline for whole-process railway environment perception and offers new insights into adapting visual foundation models to domain-specific applications.
Accurate traffic scene recognition serves as a critical foundation for decision-making and safe driving in autonomous driving and intelligent transportation systems. Existing methods mostly rely on single visual data vulnerable to environmental variations, or vision-LiDAR fusion schemes with insufficient capacity to represent road topology and traffic semantic information, limiting recognition accuracy and robustness. To address these limitations, this paper proposes an end-to-end traffic scene recognition network based on the fusion of monocular camera images and corresponding road map top-down view data. We design a learnable cross-view spatial alignment module to eliminate perspective discrepancy, and a bidirectional cross-attention fusion module to enable deep bidirectional interaction between visual semantic and map topology features. Experiments on a self-built dataset covering five typical traffic scenes show that the proposed method achieves an overall recognition accuracy of 92.6%, outperforming the best single-input baseline Swin-Tiny by 3.3%. Ablation studies further validate the effectiveness of each core module.
ZhenYu Cheng, Haoyu Kon· International Conference on...· 0 citations
Objectives: To develop a reliable multi-vehicle detection-based tracking framework under unregulated traffic scenario using enhanced vision transformer model that can precisely localize and track the vehicles. Method: The proposed framework involves a three-level enhancements, (1) Adaptive frame slicing technique in the input data loader to capture the discriminative features of near and far away vehicles in the video frame (2) Geometric positional encoding to provide clear spatial cues and fusion of features, allowing the model to better separate and detect multi-scale vehicles (3) Dual inferencing of sliced and full frame with DIoU-NMS post-processing technique to remove redundant detections. The proposed framework is evaluated on our custom developed Active Learning based vehicle dataset, “AU-INV-P-PALS”, and two public traffic datasets- “IITM-HeTra and Udacity-Fixed small”. Findings: In comparison with the existing baseline DETR model, the proposed model with SGD optimization showed 9.2% increase in the average precision for IoU = 0.5:0.95 indicating its precise localization ability. The model localizes vehicles that occupy as little as 0.2% of the surveillance video frame area. The observed tracking accuracy on real-time traffic CCTV videos indicates that the model can adapt to unregulated traffic environment and achieved appreciable score of about 85.4%, maintaining the tracking stability. The average F1-score of the enhanced DETR obtained for the test frames indicate the generalization ability of the model across different frames. Novelty: The proposed work introduces adaptive frame slicing scheme in the input data loader and geometric positional encoding which enables the detection of faraway vehicles with high accuracy in wide area surveillance imagery. Further, the enhanced model is trained using active learning based custom developed vehicle dataset that significantly reduces the reliance on massive datasets.
Keywords: Deep Learning, Active Learning, Unregulated Road Traffic Control, Vision Transformer
M. Ilamathi, Sabitha Ramakrishnan· Indian Journal of Science an...· 0 citations
To address the challenges of construction vehicle detection caused by complex illumination and severe occlusion in transportation infrastructure scenarios, as well as the requirements for emergency rescue under sudden hazards, this paper proposes a detection system based on a multi-model decision fusion architecture. The system adopts a “coarse-screening and fine-inspection” strategy, utilizing YOLOv8 for rapid global screening to ensure real-time performance. For regions with low confidence, an improved DPFL-RetinaNet is introduced to perform secondary refined detection. Concurrently, this paper proposes the Decoupled Polynomial Focal Loss based on Taylor series expansion, which enhances the mining capability for weak targets through gradient reshaping. Experimental results demonstrate that the proposed system achieves an AP50 of 87.1% on the TCVD dataset, representing an improvement of nearly 30 percentage points over the standalone YOLOv8. This provides an efficient visual perception solution for construction safety and emergency rescue in complex environments, effectively balancing detection precision and inference speed. The proposed Confidence-Guided Cascade Decision System (CG-CDS) effectively addresses the inherent trade-off between computational efficiency and detection accuracy. By dynamically routing challenging samples to a specialized fine-grained detector, the system achieves robust performance across diverse environmental conditions including low illumination, severe occlusion, and complex background clutter. Extensive ablation studies confirm that each component of the cascaded architecture contributes positively to the overall detection performance, with the proposed Decoupled Polynomial Focal Loss (DPFL) providing the most significant improvement for weak target mining.
To address the poor adaptability to complex scenes, blurred boundary details, and the difficult trade-off between accuracy and computational cost in road segmentation from unmanned aerial vehicle (UAV) imagery, this paper proposes DACS-Net, a lightweight road segmentation model based on U-Net and enhanced by dynamic attention and cross-scale semantic information. The model adopts EfficientNetV2-S as a lightweight and efficient encoder, using compound scaling and MBConv bottleneck structures to capture multi-scale features accurately. A dynamic global-local attention module (GLA) is designed to jointly model global context and local neighborhood details through a dynamic global attention branch and an adaptive local branch. Furthermore, a cross-scale semantic feature fusion module (CSF) is constructed to perform intelligent selection and efficient fusion of multi-stage encoder features through semantic-guided weight allocation, edge-enhanced feature alignment, and lightweight channel compression. Experiments on the public AeroScapes dataset show that the proposed method outperforms mainstream models in Dice, Kappa, and other metrics while maintaining relatively low computational complexity, demonstrating its effectiveness and practical value.
Wen-Wei Xiao, Yan Luo, Junlian Liao et al.· 2026 8th International Confe...· 0 citations
Reliable traffic sign detection is a prerequisite for the global deployment of autonomous driving systems, where regulatory compliance and road safety depend on perceiving signs correctly across regions, ranges, and weather conditions. Despite recent progress, vision-based methods continue to face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges (traffic signs occupy as little as $10{\times}10$ pixels at 200m), and fragile temporal tracking under the strongly non-linear perspective distortion that occurs as a vehicle approaches a sign. In this paper, we address the problem of robust, long-range, region-agnostic traffic sign perception by combining camera and Light Detection and Ranging (LiDAR) sensing. We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance. We further introduce a dual motion-model tracker that explicitly accounts for non-linear perspective transformations during vehicle approach, substantially improving temporal consistency over linear motion assumptions. Additionally, we develop a semantic attribute classification pipeline that estimates occlusion level, readability, sign embeddedness, and road relevance, providing actionable context to downstream planning. Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (OMR) of 0.49% across 221,068 evaluation sequences, demonstrating globally generalizable traffic sign perception in commercial-grade autonomous driving systems.
Meda Lazar, S. Sridhar, Shashwata Gupta et al.· 0 citations
The detection of foreign objects in key components of high-speed trains is critical for railway safety, but existing methods struggle under low-light conditions, complex backgrounds, and small objects. To address these issues, we propose an efficient detection framework integrating an adaptive brightness enhancement network (ABEN) and a lightweight train foreign object detection network (LTFD-Net). ABEN adaptively enhances images according to their illumination, improving clarity across multiple objects and backgrounds while ensuring real-time processing. LTFD-Net combines a lightweight backbone with a multi-dimensional feature enhancement module, capturing multi-scale and contextual features to accurately detect small and complex defects with minimal computational overhead. To support realistic evaluation, we introduce the high-speed train foreign object detection (HTFD) dataset with 3,904 annotated images across five key components. Experiments show that the integrated framework achieves 85.4% mean average precision (mAP) and 112 frames per second (FPS) on HTFD, surpassing state-of-the-art methods. Independently, LTFD-Net reaches 78.3% mAP and 97 FPS on NEU-DET, demonstrating preliminary generalization capability without illumination enhancement. This work provides an efficient and practical solution for real-time foreign object detection under challenging railway conditions.
Hui Lin, Junqi Li, Baolin Liu et al.· Railway Engineering Science· 0 citations