Optimization of Drowsiness Detection Using Eye, Mouth, and Hand Detection System On Raspberry Pi

Abstract

Driver drowsiness and fatigue-related crashes remain a major public-safety concern, including in D.I. Yogyakarta, where traffic accidents increased from 4,559 cases in 2020 to 7,830 in 2022. Previous Raspberry Pi 4B–based work using Haar Cascade, Dlib 68-point landmarks, Eye Aspect Ratio (EAR), and Lip Distance provided buzzer and Telegram alerts, but its evaluation was qualitative and it could not recognize yawning when the driver’s mouth was covered by a hand. This study presents an early-stage, occlusion-aware image-based validation of an alternative drowsiness detection approach rather than a completed embedded Raspberry Pi optimization system. The proposed method combines MediaPipe Face Mesh and MediaPipe Hands: facial landmarks are used to compute EAR and Mouth Aspect Ratio (MAR), while hand landmarks identify hand-to-mouth occlusion during yawning. This enables four detection classes: Normal, Eyes Closed, Yawn, and Yawn Hand Cover. Evaluation was conducted under controlled conditions using 80 manually labeled facial images, with 20 samples per class, on a laptop/PC environment; Raspberry Pi 5 deployment and real-time embedded benchmarking remain future work. The system correctly classified 75 of 80 samples, achieving 93.75% overall accuracy, with per-class accuracies of 100%, 95%, 90%, and 90%, respectively. It also produced precision, recall, F1-score, and confusion matrix analysis. These findings provide preliminary evidence that MediaPipe-based facial and hand landmarks can improve recognition of occlusion-related yawning, but broader video-based and embedded evaluations are required before claims of real-world driver-monitoring reliability can be made.