Skip to content

Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

A novel Reliability-Aware framework for MMEA (RA-MMEA), which assesses visual reliability and adaptively improves unreliable visual representations for robust entity alignment and enables more reliable visual representations for modality fusion, thereby improving performance.

Abstract

The visual modality, i.e., images, plays a key role in multi-modal entity alignment (MMEA). Existing approaches often directly fuse the image with other modalities to align different entities. Although simple, such strategies overlook the potential noise in the images and their semantic misalignment with corresponding entities, resulting in suboptimal fusion and degraded performance. Addressing this, we propose a novel Reliability-Aware framework for MMEA (RA-MMEA), which assesses visual reliability and adaptively improves unreliable visual representations for robust entity alignment. The core lies in two modules, including dependency-aware visual reliability prediction (DA-VRP) and stability-regularized visual embedding generation (SR-VEG). The former aims to estimate the reliability of an image by leveraging multi-modal dependency within the entity, while the latter focuses on producing alternative visual representation conditioned on semantics encoded in textual modalities for multi-modal fusion. Compared to current methods, RA-MMEA enables more reliable visual representations for modality fusion, thereby improving performance. In extensive experiments, RA-MMEA achieves state-of-the-art results, verifying the importance of reliable visual modality for entity alignment and the effectiveness of RA-MMEA. The code and results will be released.

View source

Similar papers

Open access 2026

Semantic-Aware Consistency Rectification for Cross-Modal Person Retrieval

This approach introduces a Paired-Guided Consistency Rectification (PCR) module that utilizes strictly paired data as reliable anchors to dynamically calibrate the confidence of unpaired samples, effectively suppressing noise from inconsistent associations under mild-to-moderate corruptions.

Zi-Xuan Zhang, Lu-Ming Xiao · 0 citations
Preprint Oct 2026

Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration

Existing multi-modal image fusion (MMIF) methods generally face an optimization gap between the visual and semantic domains, limiting the enhancement of perception capabilities. To overcome this long-standing barrier, this work proposes W2F, a new visual-semantic mutual reinforcing paradigm for MMIF. First, we reinforc...

Hao Zhang, Xu-Hui Zuo, Jia-Yi Ma · 0 citations
Open access Aug 2026

TE-CUME: Uncertainty-Aware Tri-Modal Video Retrieval Representation Learning

UAT-InfoNCE is proposed, a precision-weighted contrastive objective that adaptively modulates gradient contributions by the model’s own confidence, together with pairwise cross-modal disagreement terms that serve as zero-shot diagnostics for broadcast failure modes such as A/V desynchronisation, dubbed content, and adv...

Sanjib Kumar Swain, Santosh Kumar Swain · 0 citations
Sep 2026

Beyond Spatial-Domain Supervision: A Relation Constrained Space for Multi-Modal Image Fusion

Multi-modal image fusion (MMIF) aims to form a single image by integrating shared information, preserving complementary cues, and coordinating cross-modal conflicts across modalities. However, due to the absence of ground-truth fused images, existing MMIF supervision commonly uses spatial-domain sources or gradient var...

Ze-Yu Wang, Ming-Yu Ge, Hai-Yu Song et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Dynamic Alignment and Calibration for Multimodal Learning

Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existing methods still suffer from two limitations: (i) static cross-modal alignment strategies usually impose uniform constraints on all samples while overlooking sample-wise va...

Jing-Hao Xu, Zhen-Hua Guo, Xiao-Feng Zhu et al. · 0 citations
Oct 2026

Dual Uncertainty-Aware Guidance for Test-Time Visual Retrieval Under Distributional Variations.

Visual retrieval systems remain highly vulnerable to test-time distribution shifts caused by corruption, degradation, and environmental changes. Unlike classification, retrieval-time adaptation must preserve ranking consistency in a similarity space, where the dominant failure mode is not only inaccurate prediction con...

Yuan-Peng He, Li-Jian Li, Lin-Yu Li et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.