This work introduces SON-1K, a comprehensive benchmark for text-to-image generation, and proposes a new approach, the enhanced LMDpp, enhancing the performance of the novel two-stage Large Language Model (LLM)-grounded diffusion model pipeline (LMD).
Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.
Luecheng Wang, Shintaro Ozaki, Hidetaka Kamigaito et al.· 0 citations
A comprehensive external and internal investigation of multimodal in-context learning on the image captioning task is conducted, revealing both how ICEs configuration strategies impact model performance through external experiments and characteristic typical patterns through internal inspection.
Li Li, Yongliang Wu, Jingze Zhu et al.· IEEE Transactions on Pattern...· 0 citations
VLTok is a novel 1D hybrid tokenizer that unifies V isual and L anguage representations in a shared Tok en space through a self-prompted training paradigm, and achieves state-of-the-art performance in both image reconstruction and image generation.
Hualiang Wang, Siming Fu, Wei-Nan Jia et al.· 0 citations
It is shown that VQA can serve as an effective semantic feedback to significantly enhance prompt-image alignment without retraining the diffusion model, providing a powerful, interpretable and self-correcting strategy for text-to-image production.
DynEval, a Dynamic Evaluation framework designed to jointly assess text-to-image alignment and image quality of T2I models, is introduced and a compact evaluator achieves a higher overall correlation with human judgments.
Shyam Marjit, Dheeraj Baiju, Anuj Shikarkhane et al.· 0 citations
Understanding newspaper images remains a challenging task due to their complex, nested hierarchical structures and dense, heterogeneous layouts. In this paper, we explore two complementary approaches for newspaper structure understanding. First, we present a modular bottom-up pipeline that combines state-of-the-art open-source models: YOLO for layout detection, LayoutReader for reading order prediction, and a custom algorithm for article segmentation. This approach leverages existing robust components while maintaining flexibility and interpretability. Second, we introduce Tiramisu (Tiered Transformers for Hierarchical Structure Understanding), a novel end-to-end transformer-based architecture that explicitly models document hierarchy through an iterative tiered process. Tiramisu performs section and article separation, block localization, semantic categorization, and reading order prediction using highly parallelized attention mechanisms. Finally, we release Finlam La Libert\'e, a new dataset designed specifically for evaluating hierarchical information retrieval in historical newspapers. Experimental results demonstrate the effectiveness of both approaches in reconstructing complex newspaper hierarchies, with comparative analysis highlighting their respective strengths for scalable document digitization. The Tiramisu training code, including the synthetic newspaper generator, is available at https://git.litislab.fr/tiramisu/tiramisu-newspaper-articles-extractor.
William Mocaër, Solène Tarride, Thomas Constum et al.· 0 citations
