Hybrid Deep Learning for Individual Tree Crown Delineation and Species Classification Using UAV Imagery and Airborne LiDAR

Abstract

This study developed a hybrid deep learning framework for individual-tree crown delineation and species classification in broadleaf-dominated mixed forests using unmanned aerial vehicle (UAV)-derived multisource imagery and airborne Light Detection and Ranging (LiDAR) data. The study was conducted in two forest sites located in eastern Hokkaido, northern Japan. For individual-tree crown delineation, a Mask R-CNN model integrating UAV-derived RGB imagery and airborne LiDAR-derived canopy height model (CHM) data was developed, achieving F1-scores above 0.80. The delineated individual-tree crowns provided the basis for the subsequent species classification stage. For species classification, a deep neural network (DNN) was developed using feature embeddings extracted from the DINOv2 Vision Transformer (ViT-Small), combined with spectral and structural features derived from Normalized Difference Vegetation Index (NDVI), Green NDVI (GNDVI), and Normalized Difference Red Edge (NDRE), near-infrared (NIR), and CHM datasets. The proposed DINOv2-based DNN classifier achieved weighted F1-scores above 0.80 and outperformed the baseline Mask R-CNN-based classification approach. These results demonstrate the effectiveness of separating crown delineation and species classification tasks within a hybrid framework for accurate individual-tree species classification in complex mixed forests.