Skip to content

Early Fusion with Contrastive Learning: A Lightweight Alternative for Multi-modal Classification

2026 · International Conference on Language Resources and Evaluation · pp. 9129-9138 · 0 citations · 34 references
Computer Science

TL;DR

This work proposes a simple yet effective lightweight framework explicitly designed for multimodal classification tasks, utilising the early fusion method combined with a contrastive learning approach, which achieves performance comparable to the most competitive heavyweight state-of-the-art models and even outperforms them.

View source

Similar papers

Open access Oct 2026

Heterogeneous Multimodal Fusion Using Deep Learning for Improved Feature Extraction and Benchmark Evaluation

: Multimodal learning has emerged as a critical paradigm in artificial intelligence, enabling systems to integrate information from diverse data sources such as text, images, audio, and video. However, effectively fusing heterogeneous modalities remains challenging due to differences in feature dimensions, semantic rep...

K. Bharathi, G. S. Reddy · 0 citations
Open access Sep 2026

Cross-Modal Representation Learning for Integrating Heterogeneous Data in AI Systems

A cross-modal representation learning framework that aligns heterogeneous modalities within a shared latent representation space and exhibits strong robustness under missing modality conditions, with significantly lower performance degradation compared to baseline approaches is proposed.

I. Ibrahim, Marwan Alshar'e, I. Sanjaya et al. · 0 citations
Review Open access Oct 2026

A comprehensive review on recent pretrained multimodal deep learning models from architectures to future directions

A comparative analysis of recent pre-trained MMDL models, examining their specific applications, architectural designs, datasets utilized, and evaluation metrics used in current research to provide a clear view of current opportunities and capabilities in multimodal deep learning.

Azhar A. Hadi, K. Supreethi · 0 citations
#machine learning Preprint Aug 2026

Fusion Anything: A Generalized Multimodal Foundation Model

The Fusion Anything Model (FAM) is proposed, a foundation model for generalized multimodal data fusion that encodes transferable multimodal correlations during training and activates appropriate associations through in-context examples during inference.

Hui-Zi Cui, Zong-Bo Han, Chen Ding et al. · 0 citations
Oct 2026

Exploring Multimodal Adapters with Multi-Task Decoders for Video Action Recognition.

Large-scale vision-language pretrained models like CLIP, paired with parameter-efficient fine-tuning techniques, have emerged as promising solutions for image-to-video transfer in video action recognition. However, existing methods often prioritize strong supervised performance at the expense of transferability and gen...

Meng-Meng Wang, Ze-Yi Huang, Bo-Yuan Jiang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs

Multi-modal large language models (MLLMs) achieve strong modality understanding by pairing a large language model (LLM) with an encoder for a target modality such as vision, video, or audio. However, improving an MLLM's capability for a given modality typically requires additional training on large modality-specific da...

H. Seo, Byung Hyun Lee, Minjun Kim et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.