A survey of small object detection and tracking in UAV aerial images: from CNN to Transformer

Abstract

Small object detection and object tracking in unmanned aerial vehicle (UAV) aerial imagery constitute highly challenging research topics in the field of computer vision. These tasks are confronted with multiple difficulties, including limited object pixel occupancy, blurred textures, complex backgrounds, dramatic scale variations, and constrained computational resources. This paper systematically reviews recent advances in deep learning-based small object detection and object tracking techniques for UAV aerial imagery. For object detection, the evolution from traditional handcrafted feature-based methods to deep learning end-to-end frameworks is reviewed, with particular emphasis on two major technical paradigms based on Convolutional Neural Networks (CNNs) and Transformers. Recent optimization strategies are further summarized from three perspectives: model efficiency, object information perception, and feature fusion. For object tracking, existing approaches are categorized into two technical paradigms according to network architecture characteristics: CNN–Transformer hybrid architectures and fully Transformer-based architectures. Furthermore, hybrid architectures are classified into three categories, namely module replacement, post-feature fusion, and collaborative modeling. For fully Transformer-based methods, the evolutionary trend from static computation to dynamic computation is systematically analyzed. In addition, mainstream UAV aerial image datasets and evaluation metrics, including DroneVehicle, CODrone, and WebUAV-3M, are summarized. Comparative experimental analyses are conducted to evaluate the advantages and limitations of different algorithms. Finally, this paper discusses the ongoing transition in the field from maximizing detection/tracking accuracy toward the collaborative optimization of accuracy and efficiency, identifies key existing challenges, and outlines future research directions from the perspectives of lightweight deployment, multimodal fusion, and large-model transfer, thereby providing a systematic technical reference for related studies.