Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning

Balanced Multimodal Federated Learning: An Efficient and Noise-Resilient Approach

The metrics of individual modality contribution (IMC) and multimodal synergistic gain (MSG) are introduced to quantify sample-level and semantic-level utility, so as to guide semantic denoising selection and robust conditional balancing strategies, effectively mitigating noise interference.

IoT Data Analytics Hybrid AutoML Framework with Energy Awareness in Dynamically and Adaptively Changing Environments

The high rate of Internet of Things (IoT) deployments has contributed to high volumes of heterogeneous data that are generated under drastic conditions of energy and computing constraints. The paper is a fully energy-saving IoT data analytics framework viewed through the prism of Automated Machine Learning (AutoML). The described comparison between classical machine learning models (ML), deep learning models (DL), and hybrid models (using an ensemble) evaluates the models using metrics where the scores are multi-dimensional, i.e., they consist of such measures as accuracy, power consumption and energy efficiency, inference latency, CPU usage, and memory consumption. An experimental evaluation of a practical smart city dataset of IoT demonstrates that there is a pronounced trade-off between the predictive performance and power expenditure. Deep learning models are also power-intensive despite their accuracy being high. To surpass that, a hybrid ML-DL mixture composed of CNN, SVM, and Random Forest ones is proposed in the assistance of energy-related weighted averaging. The power consumption of the hybrid model is 28.18 W, and its accuracy is 0.897, which offers the trade-off of using the hybrid model in the resource-constrained and dynamic Internet of Things.

Edge–Cloud Collaborative Deep Learning Framework for Real-Time IoT Analytics

The Internet of Things (IoT) ecosystem has expanded and produced massive volumes of real-time streaming data, that must be processed quickly and in an efficient way. The latency problem, bandwidth congestion and bandwidth limitations in the traditional cloud-centric processing architecture are significant challenges, particularly in latency-sensitive systems such as smart healthcare, industrial automation and intelligent transportation systems. While edge computing reduces the amount of data that needs to be sent, edge devices are typically more limited in resources, which means they cannot run complex deep learning models. To address these limitations, this paper proposes a new Edge–Cloud Collaborative Deep Learning (ECC-DL) Framework to orchestrate different aspects of inference and learning between the edge nodes and cloud server. The lightweight convolutional neural networks (CNNs) are deployed at the edge nodes to extract features and make initial inferences, and the heavier end models are deployed at the cloud end for global analysis and regular retraining. To optimize the partitioning of the workload between the resources, an adaptive workload partitioning algorithm is designed that considers latency, bandwidth and computational resources. By experimental evaluation, latency is reduced by 40% and bandwidth consumption is reduced by 30% and prediction accuracy is improved over the centralized approach.

Energy-efficient federated deep learning for edge-based image recognition in resource-constrained IoT environments

Cross-Modal Transformer Networks for Unified Intelligence in Multi-Sensor IoT Systems

The emergence of multi-sensor Internet of Things (IoT) deployments has led to an immediate demand to have unified intelligence models that can take advantage of heterogeneous data modalities, such as time-series sensor readings, visual data, audio streams as well as contextual metadata, to form incoherent understanding and decision-making. The conventional methods treat each of the modalities separately, which does not reflect the deep cross-modal associations that are necessary to understand a scene holistically and act appropriately in a context. The proposed paper suggests a new cross-modal transformer architecture network operating on the principle of joint representation learning of various sensor modalities in order to achieve the unification of intelligence in multi-sensor IoT systems. The framework presented is based on modality-specific encoders and then a set of cross-modal attention mechanisms which allow two way flow of information between sensor streams to capture the highly complex inter-modal dependencies without paired training data. The new hierarchical fusion approach is a combination of the local cross-modal interactions and global contextual reasoning where the model can adaptively weigh the contributions of the sensors depending on the environmental conditions and tasks. The architecture requires built-in adaptive modality gating mechanisms, which ensures that performance does not suffer even in case of a failure of individual sensors or in case of poor quality. Cross-modal evaluation on three multi- sensor IoT benchmark datasets indicates that the proposed cross-modal transformer is 96.8% accurate in unified perception tasks, which is 18.4% and 11.2% better than single-modality evaluation and other traditional fusion methods, respectively. The framework exhibits high levels of robustness to sensor dropouts of up to 40 percent and is also able to generalize to unobservable sensor layouts. The results have made cross-modal transformer networks a disruptive paradigm of unified intelligence in heterogeneous multi-sensor IoT systems.

Multimodal Sensor Data Fusion for Robust Deep Learning Models in Manufacturing Applications