A Benchmark for Human Personality Analysis in Multimodal Dialogues
Abstract
Human personality analysis is crucial for personalized dialogue systems, mental health assessment, and human–computer interaction, but existing resources are mostly text-only and fail to capture rich cognitive cues (e.g., speaker-awareness and emotions) in real conversations. We present a new benchmark for human personality analysis in multimodal dialogues (HPAMD), built on 218 real-world IELTS-style interview videos (RIIV) and segmented into 2,322 turn-level units with aligned emotion-related dialogue text, acoustic, visual, and continuous Big Five annotations. We further propose DAMP, a dialogue-aware multimodal framework that models turn-level question-response interactions and integrates emotion reasoning for personality prediction. Experiments show dialogue modeling outperforms whole-video modeling, emotion-enhanced representations bring additional gains, and DAMP achieves the best overall performance under the full setting. The dataset is publicly available at: https://github.com/lulvhui/RIIV.