mmSketcher: Physics-Grounded and Editable Scenes for mmWave Human-Sensing Simulation

Abstract

Learning-based millimeter-wave (mmWave) human sensing needs data spanning environments, radar placements, people, and activities, yet each dataset is coupled to the deployment that produced it, and rare or unsafe events are the hardest to capture. Simulation can vary these factors, but existing mmWave simulators hard-wire them into task-specific pipelines, making coordinated edits hard to express and verify. We present mmSketcher, a scene-first middleware whose single editable scene is synchronized behind replaceable physics and RF engines. Moving the radar, replacing a person's motion, or adding an occluder is one edit to this shared state, and path geometry, Doppler, occlusion, and labels update together. Motion-capture and monocular-video humans share one deforming mesh whose per-face velocities drive Doppler, and each radar frame yields raw complex time-multiplexed multi-antenna (TDMA-MIMO) measurements. We validate the abstraction with analytic scene-edit checks, a matched real capture against native RF-Genesis, a fixed-budget substitution pilot on the HuPR pose benchmark, and throughput measurements: fidelity is comparable to the baseline with different tradeoffs, simulated sequences can stand in for part of a real training budget, and generation runs at offline-to-interactive rates. With the scene as a reusable control plane, a new sensing question becomes a new experiment rather than a new simulator. We release our evaluation artifacts on GitHub.1