Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs
DMC-Repair is proposed, which trains models on the same kind of cross-modal swapped samples while assigning supervision according to the modality specified by the question, and reduces the image-induced share of the answer effect by 59.9%, effectively suppressing the cross-modal shortcut without compromising audio-ques...