Knowledge-guided machine learning in computer vision: A systematic survey and taxonomy (2014-2025)
TL;DR
A systematic survey of KGML in computer vision based on a PRISMA-guided review of studies published between 2014 and 2025 and proposes a taxonomy that distinguishes between scientific knowledge and non-scientific knowledge, and further categorizes knowledge integration strategies into knowledge-guided learning, knowledge-guided architectures, and knowledge-guided pre-training.
Abstract
Knowledge-Guided Machine Learning (KGML) has emerged as an important paradigm for addressing the limitations of purely data-driven models by incorporating domain knowledge into the learning process. Although previous surveys have discussed KGML in broad terms, a comprehensive review dedicated specifically to computer vision remains lacking. To address this gap, this paper presents a systematic survey of KGML in computer vision based on a PRISMA-guided review of studies published between 2014 and 2025. From an initial pool of 2,788 publications, 286 high-quality studies were selected for detailed analysis. Based on this review, we propose a taxonomy that distinguishes between scientific knowledge and non-scientific knowledge, and further categorizes knowledge integration strategies into knowledge-guided learning, knowledge-guided architectures, and knowledge-guided pre-training. The survey shows that KGML has been applied across a wide range of computer vision tasks, including classification, object detection, segmentation, and multimodal understanding, with reported benefits in generalization, interpretability, and robustness. In addition, we identify key open challenges related to knowledge representation, fusion complexity, evaluation, and bias, and outline future research directions for advancing KGML in computer vision.