Skip to content
Log in
Preprint

Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

Boxin Chang Yu-Chih Chen
Jul 2026 · 0 citations · 33 references
Computer Science

TL;DR

An Implicit Cultural Alignment Reward Model built upon a lightweight 4.2-billion-parameter Multimodal Large Language Model (MLLM) integrated with a Skip-connection Cross-Attention mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details.

Abstract

As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI. Existing T2I evaluation metrics and multimodal judges often rely on visual-semantic representations that underrepresent implicit cultural norms, leading to biased preference judgments and the omission of fine-grained cultural cues. In addition, visual question answering (VQA)-based evaluators typically depend on autoregressive text generation, which limits their scalability for real-time reward modeling. To address these limitations, we introduce an Implicit Cultural Alignment Reward Model built upon a lightweight 4.2-billion-parameter Multimodal Large Language Model (MLLM). Our framework integrates an Implicit Cultural Probe with a Skip-connection Cross-Attention (SkipCA) mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details. Evaluations on 3,323 challenging and carefully curated image pairs from the CulturalFrames benchmark show that our approach achieves 82.12% pairwise accuracy, with Pearson and Kendall correlation coefficients of 0.585 and 0.412, respectively, outperforming representative vision-language metrics and MLLM-based evaluators. Moreover, by bypassing autoregressive text generation, our model processes each evaluation in 0.21 seconds under our local inference setup, achieving a $10\times$ speedup over standard VQA-based evaluators. These results suggest that the proposed reward model can provide an efficient and culturally aware scalar signal for preference optimization pipelines such as Reinforcement Learning from Human Feedback and Direct Preference Optimization. Additional resources are available on our project page at https://bensonch1214.github.io/Implicit_Cultural_Alignment/.

arXiv Semantic Scholar
View source

Similar papers

Preprint Jul 2026

Evaluation-Verification Reward for Consistent Multi-Reference Image Editing

While recent image editing models have made rapid progress, multi-reference editing remains challenging, particularly in maintaining visual consistency across references and ensuring overall visual harmony. Reinforcement learning has proven highly effective for text-to-image generation and single-image editing, but its extension to multi-reference editing is hindered by the absence of suitable reward models that capture multi-image relational constraints. Moreover, naively using multimodal large language models(MLLMs) as zero-shot evaluators faces a key tension between hallucination-prone long-form reasoning and the limited deductive power of short-form judgments. We address these issues with a Multi-dimensional Evaluation-Verification Reward(EVR). EVR decomposes evaluation into distinct visual criteria; for each criterion, an MLLM Evaluator generates multiple candidate hypotheses, and a Verifier grounds each claim in concrete visual evidence to accept or reject it, producing reliable and fine-grained reward signals. Together with a scalable data pipeline, our method enables RL fine-tuning of off-the-shelf editors without architectural changes. Extensive experiments show substantial gains over the base Qwen-Image-Edit, improving consistency and harmony to match or surpass NanoBanana.

Yingmao Miao, Pengfei Zhang, Xiaochen Lv et al. · 0 citations

Prompt Yourself: Awakening Textual Semantics in 1D Visual Tokenizers

VLTok is a novel 1D hybrid tokenizer that unifies V isual and L anguage representations in a shared Tok en space through a self-prompted training paradigm, and achieves state-of-the-art performance in both image reconstruction and image generation.

Hualiang Wang, Siming Fu, Wei-Nan Jia et al. · 0 citations
Preprint Aug 2026

CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation

Text-to-video (T2V) generation models have advanced rapidly, yet their ability to represent diverse cultural contexts remains underexplored. Existing benchmarks mainly focus on perceptual quality, physical plausibility, and text-video alignment, but do not directly assess whether generated videos capture culturally specific objects, actions, rituals, visible text, or audio cues. We introduce CultureVidBench, a comprehensive benchmark for evaluating cultural understanding in T2V generation. CultureVidBench contains 1,000 curated prompts covering 12 countries, 6 continents, 8 cultural regions, and 14 cultural aspects organized into three categories: material culture, social practice&performance, and ritual&ceremony. Designed specifically for video generation, CultureVidBench emphasizes dynamic and multimodal cultural representation, including social interactions, ritual procedure, and culturally appropriate visible text and audio. We evaluate seven representative T2V models through human user studies and MLLM-based automatic assessment across cultural faithfulness, multimodal cultural rendering, semantic adherence, and perceptual quality. Results show that although current models achieve strong semantic adherence and visual quality, they often fail to faithfully capture fine-grained cultural details, particularly for underrepresented regions, rituals, and multimodal cultural cues.

Xianjing Han, Yuhan Su, Yang Deng et al. · 1 citation
Preprint Jul 2026

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

MonkeyOCRv2, a visual-text pretrained model for document AI, is presented, and a pretraining strategy that jointly learns image-to-text generation and pixel-level document reconstruction is proposed: the former aligns visual representations with textual content, while the latter preserves character strokes and layout details.

Yuliang Liu, Zhang Li, Ziyang Zhang et al. · 1 citation