CCS: a continuous spatial-semantic concordance score for robust evaluation of object detection models

Abstract

Standard object detection evaluation relies on hard-threshold metrics (mAP, F1) that collapse continuous localization into binary decisions, penalize near-misses as complete failures, and ignore semantic label hierarchies. We propose CCS, a Continuous Spatial–Semantic Concordance Score integrating two complementary terms: (i) Csp\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{\text {sp}}$$\end{document} models bounding boxes as 2D Gaussians and computes closed-form normalized cosine similarity, smoothly penalizing spatial deviation without arbitrary IoU cutoffs; (ii) Csem\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{\text {sem}}$$\end{document} leverages taxonomic Wu–Palmer similarity to award partial credit for clinically adjacent misclassifications (e.g., “red-spotted tongue” vs. “red tongue”). We formulate both a lightweight class-level score and an instance-level extension (CCSinst\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{CCS}_{\textrm{inst}}$$\end{document}) via global Kuhn–Munkres bipartite matching. Benchmarking six detectors across three YOLO generations (v8/v10/v11) on an eight-class tongue-symptom dataset with severe difficulty imbalance, hard-threshold F1 degrades sharply (up to 16.9%) and reverses rankings for 33% of detector pairs, while CCS remains stable (0.62−0.65) and preserves overall ranking (Spearman ρ=0.94\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho = 0.94$$\end{document}, Bootstrap 95% CIs, Hedges’ g≤0.121\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g \le 0.121$$\end{document}). To evaluate operating-point dependence, we define continuous Mean CCS (mCCS\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mCCS}$$\end{document}) and threshold-integrated AUCCCS\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{AUC}_{\textrm{CCS}}$$\end{document} across confidence thresholds τ∈[0.1,0.9]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau \in [0.1, 0.9]$$\end{document}, demonstrating robust ranking preservation alongside NMS invariance (σ<0.0003\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma < 0.0003$$\end{document}). Furthermore, evaluation across four hierarchical taxonomies yields invariant model rankings (ρ=1.0000\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho = 1.0000$$\end{document}), confirming metric robustness against hierarchy variations. Finally, CPU latency benchmarking confirms extreme efficiency: class-level CCS incurs only 0.007 ms/image (>11×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$>11\times $$\end{document} faster than full COCO mAP at 0.079 ms), while instance-level Hungarian matching operates in real time at 0.022 ms/image (46, 239 FPS). These results establish CCS as an efficient, threshold-independent evaluation protocol for class-imbalanced, semantically structured medical detection tasks.