Edge AI Deployment of Lightweight TCN-SE for Real-Time Sleep Apnea Detection Using SpO₂ Signal: A Cross-Platform Deployment Case Study

Abstract

Obstructive sleep apnea (OSA) affects nearly one billion people worldwide, yet diagnosis remains bottlenecked by labor-intensive polysomnography. This study presents a cross-platform deployment case study of a lightweight Temporal Convolutional Network with Squeeze-and-Excitation attention (TCN-SE, 183K parameters), used as a representative compact fully convolutional model, for per-minute sleep apnea classification from a strictly causal, trailing 10-minute SpO2 window. The model was developed on 997 subjects from the Sleep Heart Health Study (SHHS; 5-fold subject-level cross-validation AUC <inline-formula> <tex-math notation="LaTeX">$0.9025~\pm ~0.0028$ </tex-math></inline-formula>) and evaluated on all remaining eligible SHHS-1 recordings — a strictly held-out, natural-prevalence cohort of 4,796 subjects (2.38 million minutes) — as well as on all 2,056 eligible recordings of the external MESA cohort. On the held-out cohort the model achieves a per-minute AUC of 0.896 (95% CI 0.894–0.898) and screens for clinically significant OSA with AUC 0.967 (AHI <inline-formula> <tex-math notation="LaTeX">$\ge 15$ </tex-math></inline-formula>) and 0.979 (AHI <inline-formula> <tex-math notation="LaTeX">$\ge 30$ </tex-math></inline-formula>), with NPV 0.997 for severe OSA at natural prevalence; external MESA screening AUCs (5-fold ensemble) are 0.962 and 0.956. We benchmark deployment on a server GPU (NVIDIA A6000), an edge GPU (Jetson AGX Orin, TensorRT), and a Korean edge NPU (Mobilint MLX-A1, ARIES2) under a unified protocol (five repeated runs, p99 latency, idle/active power, 30-minute sustained-load stability). TensorRT FP16 matches the FP32 reference to four AUC decimal places (<inline-formula> <tex-math notation="LaTeX">$\Delta $ </tex-math></inline-formula>AUC + 0.0000; maximum probability difference 0.003), whereas INT8 fidelity depends strongly on the toolchain: the NPU’s qb SDK is near-lossless (<inline-formula> <tex-math notation="LaTeX">$\Delta $ </tex-math></inline-formula>AUC −0.0005, 95% CI −0.0007 to −0.0003) and insensitive to calibration-set size, while TensorRT loses 0.0025–0.0081 AUC depending on calibration method and degrades as the calibration set grows. The NPU also delivers the lowest latency (0.239 ms mean, 0.296 ms p99), the smallest engine (273 KB), and throttle-free fanless operation, at incremental power comparable to the edge GPU (<inline-formula> <tex-math notation="LaTeX">$\Delta 3.3$ </tex-math></inline-formula> W vs. <inline-formula> <tex-math notation="LaTeX">$\Delta $ </tex-math></inline-formula>2.0–3.7 W). An architecture ablation deployed on both edge platforms shows that a GRU with marginally higher accuracy (+ 0.006 AUC) cannot be compiled for the NPU and is <inline-formula> <tex-math notation="LaTeX">$\approx 38\times $ </tex-math></inline-formula> slower on the edge GPU, supporting fully convolutional designs for NPU-based clinical monitoring.