Real-time polyp characterization using a dual-stage deep learning approach

Abstract

Colorectal cancer is one of the most common cancers worldwide and remains a major cause of cancer-related mortality. During colonoscopy, accurate identification of colorectal polyp types is important for clinical decision-making, particularly in classifying polyps as either neoplastic or hyperplastic. However, many existing approaches use a single-stage framework that detects and classifies polyps simultaneously from the full colonoscopy image, which may limit characterization performance. In this thesis, a dual-stage deep learning framework with a color-mode-aware routing module is proposed for real-time polyp characterization. The first stage detects polyp regions using YOLOv5-M, while the second stage classifies the detected regions using EfficientNetV2-S. Square cropping with black padding is applied to preserve lesion and contextual information. The routing module adapts the model to different imaging modes, including WLE and non-WLE. Evaluated on a private colonoscopy dataset annotated by expert endoscopists, the framework achieved F1-scores of 88.4% for neoplastic polyps, 65.3% for hyperplastic polyps, and a macro F1-score of 76.9%. The full pipeline achieved a latency of 19.91 ms, or 50 FPS, indicating suitability for real-time colonoscopy applications.