Missing data imputation using unsupervised learning for multi-sourced heterogeneous data in bridge structural health monitoring
TL;DR
Comparisons with the state-of-the-art methods, including K-nearest neighbour, ridge regression, and random forest, demonstrate that the mask-trained AE/VAE provides stable performance across diverse missing patterns without requiring repeated model training, offering a practical solution for heterogeneous bridge SHM data recovery.
Abstract
In the field of structural health monitoring (SHM) for bridges, missing data is a common and critical issue, often caused by sensor failures, communication interruptions, or other effects due to extreme environmental conditions. These data gaps severely impact the performance and serviceability of monitoring systems and the reliability of decision-making processes. To address this problem, this paper proposes an unsupervised learning method for missing data imputation based on autoencoders (AEs) and variational AE (VAE) tailored for multi-sourced heterogeneous bridge monitoring data. The method accounts for data heterogeneity by constructing a multi-layer AE network that learns deep features to model potential inter-sensor correlations. The data is first pre-processed into a unified format through normalization and feature extraction. A pseudo-missing mask generation strategy is applied during training to simulate various missing patterns, enhancing model robustness. Using actual monitoring data from the Forth Road Bridge (UK, Spring 2022), comparative experiments validate the method. The results show that under single-feature missing scenarios, AE and VAE achieve comparable high accuracy for wind speed and lateral deformation (R2 > 95%) and temperature (R2 > 80%), but both are challenged by complex vertical deformation (R2≈50%). In complex multiple-feature missing scenarios, particularly when all deformation data are absent, VAE demonstrates superior performance (R2 = 57% for vertical deformation) by effectively leveraging latent correlations, outperforming AE (R2 = 46%). This indicates that the deterministic AE excels in reconstructing data with simpler, linear dependencies, while the probabilistic VAE is more adept at handling intricate, nonlinear relationships. Comparisons with the state-of-the-art methods, including K-nearest neighbour, ridge regression, and random forest, further demonstrate that the mask-trained AE/VAE provides stable performance across diverse missing patterns without requiring repeated model training, offering a practical solution for heterogeneous bridge SHM data recovery.