Millimeter-wave (mmWave) radar enables privacy-preserving and illumination-robust human motion reconstruction, but training generalizable models typically requires costly paired radar-motion recordings. Simulation can scale such supervision, yet even physics-based simulators cannot fully reproduce real-world multipath, clutter, hardware-specific response statistics, or distance-dependent resolution degradation, leaving a sim-to-real gap. We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors. To learn transferable signal and motion priors, we pretrain a multimodal radar encoder with a physics-informed domain-randomization curriculum designed to mitigate the sim-to-real gap by approximating real-world propagation- and acquisition-level variations, while a joint-temporal tokenizer learns a discrete prior over plausible human motion. A dual-mode mapping module predicts either motion-code distributions for structurally constrained zero-shot reconstruction or continuous motion parameters for flexible adaptation from limited real data. We further construct a 4.2M-frame, 31K-sequence dataset suite and introduce a No-Overlap Setting that prevents any exact subject-environment-location-motion tuple from appearing in both the adaptation and test sets. Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains: with only 24 paired real sequences, mmSimPrior-Reg reduces MPJPE by 24.7-39.0% over the strongest baseline across the three environments, while mmSimPrior-Cls reduces zero-shot MPJPE by 8.5% without fine-tuning.
High-fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar-based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri-plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual-receiver geometry to fit and reproduce site-specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed-room setting show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.
Weitao Xiong, Tianyu Liu, Peng Li et al.· 0 citations
This work proposes Physics-Unrolled Hybrid Neural Operator (PU-HNO), a three-stage cascade that predicts high-fidelity indoor radio maps from low-fidelity ray-tracing outputs and scene priors by progressively capturing reflection, diffraction, and scattering effects, rather than treating radio maps as generic images.
Rafid Umayer Murshed, Saif Ur Rahman, Mingyue Tang et al.· 0 citations
Sparse and noisy millimeter-wave radar point cloud observations often correspond to multiple plausible human poses, making deterministic pose estimation fundamentally ill-posed. Yet existing radar methods remain deterministic, collapsing this ambiguity into a single estimate. Diffusion-based alternatives can model multi-hypothesis distributions but require costly sequential denoising for each distribution sample and lack calibrated uncertainty. We propose Multi-Hypothesis Normalizing Flow Pose Generator (MH-NFPG), which models pose distributions from radar point clouds using a conditional normalizing flow. Specifically, we combine a spatiotemporal transformer backbone with a normalizing flow that transforms a Laplace base distribution into an expressive posterior, generated in parallel through a single forward pass. Leveraging this efficiency, we outperform diffusion-based alternatives in calibration across three radar benchmarks (MM-Fi, mmRadPose, mRI), improve pose accuracy on two, and match it on the third, while achieving over 20x faster inference for applications and reducing calibration error by up to 85%. We find that calibration degrades substantially for diffusion models, whereas our flow-based approach maintains reliable coverage, also in cross-environment settings. These results demonstrate normalizing flows as a practical alternative to diffusion models for real-time, uncertainty-aware radar pose estimation. Our code will be made publicly available.
J. Mueller, Sebastian Hoefler, D. Zanca et al.· 0 citations
Synchronized camera and wireless measurements observe the same scene through different physical channels. The central difficulty is that a representation learned in one deployment can fail when viewpoint, traffic, illumination, and propagation geometry change. This paper presents CM-MAE, a self-supervised vision--wireless pretraining framework for cross-scenario representation transfer. The evaluated real-data model uses only RGB frames and the measured 64-beam received-power vector available in DeepSense 6G; it does not use ray-traced paths, calibrated depth, or beam-index labels during pretraining. Its central pretraining term is a \emph{soft contrastive alignment loss}. Instead of making the synchronized image--wireless pair the only positive pair, this loss builds a target distribution from similarities between measured beam-power profiles, so nonidentical samples with similar directional responses are not forced apart as false negatives. A masked joint decoder provides the complementary local objective by reconstructing hidden visual patches and wireless angular clusters under modality dropout. After pretraining, a differential-rate fine-tuning rule lets a new fusion head adapt quickly while the encoders move slowly. Under a sequence-disjoint DeepSense 6G protocol, adding the soft alignment loss improves a matched linear-probe transfer average from 24.88\% to 29.49\%. Mild fusion fine-tuning reaches 77.38\% Top-1 accuracy on unseen Scenarios 6--8, and optional transductive normalization adaptation reaches 78.69\%. Since the fusion setting uses the contemporaneous 64-beam power vector at inference, these results should be read as representation-transfer diagnostics, not as proactive beam-prediction or reduced-sweeping claims.
Reconstructing precise character trajectories in multipath environments represents a primary challenge for high-fidelity reconstruction for radar-based in-air writing. Existing tracking methods typically rely on dominant target assumptions, often failing to account for the interference of multipath reflections. Conversely, end-to-end deep learning approaches generally lack the physical interpretability necessary for accurate geometric drawing. In this work, we propose a novel tracking framework that combines high-resolution D-band sensing with a data-driven probabilistic data association (PDA) architecture. We leverage a 56 GHz bandwidth frequency-modulated continuous-wave (FMCW) radar setup to capture fine-grained kinematic signatures, supported by a rigorous sensor placement and calibration strategy. To address multipath interference without relying on rigid parametric assumptions, we propose the Temporal Deep PDA (TD-PDA). This architecture replaces heuristic hard assignments and classical parametric clutter models with a covariance-aware causal temporal convolutional network (TCN) fused directly into the statistical covariance spread equations of an extended Kalman filter (EKF). To train the network without heuristic bias, we introduce a track quality indicator (TQI) ensemble to automatically extract high-fidelity pseudo-labels. The framework is rigorously validated via simulated ground-truth data to quantify the geometric impacts of hardware constraints, and experimentally proven on a 1000-gesture dataset using tenfold leave-one-Subject-Out (LOSO) cross-validation. The proposed TD-PDA generalizes to unseen users with an ultralow inference latency, successfully reconstructing legible trajectories even in the presence of strong multipath interference. The model demonstrates improvements over prior deep-association architectures and achieves stability comparable to a well-tuned classical PDA filter via a purely data-driven design.
Salah Abouzaid, Leander Nothelle, Nils Pohl· IEEE Transactions on Radar S...· 0 citations
We introduce Simulation-Based Imaging (SBI), a framework for non-destructive acoustic imaging in which machine learning models trained entirely on simulated data serve as real-time solvers for the acoustic inverse problem. A high-fidelity nodal Discontinuous Galerkin forward solver generates large training datasets by randomizing inclusion geometry within a unit-cube domain; a 2D convolutional neural network then learns a direct mapping from boundary pressure measurements to a 32 by 32 by 32 voxel reconstruction of the interior. The trained model reliably recovers inclusion position and size from 144 boundary sensors with no prior knowledge of inclusion count or geometry. Reconstruction error degrades by only 13% under 5% additive measurement noise, and just 17% of the sensor array (24 of 144 sensors) suffices for quality within 4% of full coverage. These results establish SBI as a viable proof-of-concept imaging device whose complexity resides in software rather than hardware, opening a path toward cheap, portable, deployable imaging systems.
