A hybrid hierarchical multimodal learning with autoencoders, vision transformers and deep belief networks for liver cancer detection

Abstract

A clinical problem of early liver cancer is one of the most urgent ones due to the insidious nature of tumors, the lack of multimodal data consistency and large inter-subject variation. Conventional systems of diagnosing rely highly on manual interpretation and monomodality images that typically result in delayed diagnosis and reduced effectiveness in the treatment. The researcher in this study presents another model of multimodal deep intelligence to detect and categorize early liver cancer through the combination of magnetic resonance imaging (MRI) and computed tomography (CT) data. The provided solution is an integrated self-regularized autoencoder (AE) to learn the multimodal representation and allows noise suppression and compression of latent features. Vision Transformer (ViT) involves extracting long-range spatial dependencies and context in fused representations. Dynamic maximization of diagnostic decision policy and classification confidence under different clinical settings ML networks are A Deep Belief Network (DBN) that has hierarchical tissue abstraction and probabilistic pattern modelling and Deep Q-Network (DQN) that optimizes diagnostic decision policy and classification confidence. A standard liver imaging dataset (612 institutional patients to be assessed in the first instance and 512 public patients to be assessed in the second instance) yields high diagnostic quality, potential for generalization, and classification accuracy with positive results of 98.76% (95% CI 98.12–99.34) and an AUC of 0.992 (95% The findings confirm that there was successful multimodal fusion with heterogeneous deep learning paradigms in the detection of early liver cancer. This research will add the presence of scalable, intelligent, and clinically relevant diagnostic architecture to facilitate timely responses and decision support systems to manage liver cancer.