A Multimodal Camera-and-IoT Framework for Real-Time Crowd Monitoring, Risk Prediction, and Incident Detection

Abstract

Crowd crushes, stampedes, and fires at large gatherings kill and injure thousands each year, and the cause is almost always the same: the danger was recognised too late. Manual CCTV monitoring cannot watch dozens of feeds at once, and camera-only systems are blind to hazards that leave no visual crowd signature, such as a fire starting at the edge of a dense gathering. This paper presents a multimodal framework that fuses camera analytics with Internet-of-Things sensors to monitor a crowd, predict risk, and detect incidents in real time. From each scene it derives camera features (density, velocity, divergence, motion energy) and IoT features (acoustic level, turnstile count, smoke, temperature), fuses them, computes a weighted risk score with four response tiers, and classifies the situation into six categories - normal, congestion, panic, stampede, fight, and fire - with a Random Forest. On a labelled dataset of 3,000 scenes the system reaches 96.8 percent accuracy and a 0.968 macro F1 on the six-class task, validated at 97.6 plus or minus 0.5 over five runs, and an occlusion correction cuts crowd-counting error from a mean absolute error of 116 to 15 people. The central finding is that the modalities are complementary: a camera-only model detects fire with only 63 percent recall, while the multimodal system raises that to 99 percent. The pipeline processes a frame in 36 milliseconds (27.5 frames per second) and scales to 100 cameras.