High-precision railway track defect identification using YOLOv11: a comparative evaluation with Roboflow 3.0

Abstract

This study presents a comparative evaluation of two object detection pipelines, YOLOv11 and Roboflow 3.0, for automated railway track defect identification, assessing their detection accuracy, reliability, computational efficiency, and suitability for real-time deployment. A binary detection dataset of 934 labelled images (748 training/112 validation/74 test) representing defective and non-defective railway track conditions was used. Standardized preprocessing included automatic orientation correction, resizing to 640×640 pixels, and a six-strategy augmentation pipeline. Both models were trained under identical conditions on Roboflow's cloud GPU infrastructure. Evaluation employed mAP@50, mAP@50–95, precision, recall, F1 score, and confusion matrices. Limitations include the use of a single dataset collected under constrained conditions and the absence of field-deployment tests. YOLOv11 achieved mAP@50 of 99.5%, precision of 98.8%, recall of 99.8%, and F1 of 99.3%, with perfect class-wise accuracy on both validation and test sets. Roboflow 3.0 achieved mAP@50 of 91.3%, precision of 82.6%, recall of 85.9%, and F1 of 84.2%. These results should be interpreted with caution given the limited test set size; confidence intervals and cross-validation analyses are reported to contextualize their reliability. YOLOv11 demonstrates substantially superior fault-detection performance attributable to its refined multi-scale feature extraction, efficient detection head, and modern backbone optimization. To the best of the authors' knowledge, this is among the first studies to compare YOLOv11 directly with the Roboflow Train 3.0 pipeline on a binary railway defect detection task under fully controlled conditions.