Skip to content
Conference

Dual-path attention modulation for training-free text-guided image editing

Jul 2026 · International Conference on Generative Artificial Intelligence and Image Processing · Vol 14292, pp. 1429204 - 1429204-7 · 0 citations · 10 references
Engineering

TL;DR

A training-free Dual-path Attention Modulation (DAR) framework that decouples semantic edits while preserving source image structure is proposed and Adaptive Self-Attention (ASA) and Adaptive Cross-Attention (ACA) modules that dynamically regulate attention replacement are introduced.

Abstract

Text-guided image editing with diffusion models faces a core challenge: simultaneously executing semantic edits while preserving source image structure. We propose a training-free Dual-path Attention Modulation (DAR) framework that decouples these objectives. DAR establishes cross-image feature alignment in self-attention for spatial consistency and semantic remapping in cross-attention for precise attribute binding. To overcome limitations of fixed replacement ratios, we introduce Adaptive Self-Attention (ASA) and Adaptive Cross-Attention (ACA) modules that dynamically regulate attention replacement based on real-time visual and semantic feedback. Experiments on Wild-TI2I and ImageNet-R-TI2I demonstrate that our method achieves superior structural fidelity (SSIM: 0.7721 on Wild-TI2I, 33% improvement over PnP) while maintaining competitive semantic alignment, with high efficiency (11.3s per image).

View source

Similar papers

Open access Jul 2026

Text Image Super-Resolution via Fusion of OCR Priors and Cross-Scale Attention

Text image super-resolution aims to improve the readability of low-quality text images while preserving character structures, stroke details, and semantic consistency. Compared with natural image super-resolution, this task is more sensitive to structural distortion because small changes in stroke topology may lead to incorrect text recognition. To address this problem, this paper proposes an OCR prior-guided cross-scale framework for text image super-resolution. Specifically, character-level semantic priors extracted from a pretrained OCR model are introduced to provide structural guidance for degraded text reconstruction. A gated feature modulation mechanism is designed to adaptively regulate the contribution of OCR priors, reducing the influence of unreliable semantic predictions. A cross-scale dynamic attention module is also developed to aggregate multi-granularity visual features, enabling the model to jointly recover fine stroke boundaries and global character structures. In addition, a sequence-aware calibration module is introduced to improve structural consistency along the logical reading order of text. Experiments on mixed text image benchmarks and the TextZoom dataset show that the proposed method achieves competitive or better performance among the compared methods in terms of PSNR, SSIM, and recognition-oriented metrics. Additional ablation, OCR prior robustness, and computational complexity analyses further indicate that the proposed framework improves text readability while maintaining a reasonable accuracy–complexity trade-off. The results also suggest that OCR priors are useful for text image reconstruction, but should be used as soft constraints when external recognition predictions are uncertain.

Xinyu Qiu, Jingchao Liu, Chen Fang · 0 citations
Preprint Jul 2026

DiTailed: Ensuring Visual Object Consistency in Text-Image-to-Image Flow Matching Models

ABO-Edit is introduced, a dataset specifically designed to study object consistency, comprising over 12,000 triplets of source images, editing prompts, and high-quality target images rendered from artist-designed 3D assets, with multi-view coverage and human-verified quality control.

Francesco Taioli, Daniel Coelho, Iaroslav Melekhov et al. · 0 citations
Conference 2026

Structure-Aware and Frequency-Guided Diffusion Framework for Multimodal Fashion Image Editing

Fashion image editing aims to modify target garment attributes under textual and reference guidance while preserving non-target contents. However, existing methods often suffer from inaccurate garment localization, insufficient preservation of high-frequency textures, and unnatural transitions near edited boundaries. To address these issues, we propose a structure-aware and frequency-guided framework for multimodal fashion image editing. Specifically, we design a Structure-Guided Textual Mask Network to predict geometry-aware editing regions by leveraging refined textual structural cues and human-centric priors, where a structural prior reweighting mechanism is introduced to improve localization accuracy. We further develop an adaptive frequency-domain texture enhancement module to inject high-frequency fabric details from a reference image during late denoising, and employ a boundary-band soft fusion strategy to ensure smooth visual transitions. In addition, we construct a new dataset, DFEdit, for fine-grained multimodal fashion image editing. Experimental results show that the proposed method achieves competitive performance in terms of editing fidelity, texture consistency, and visual quality, showing its effectiveness for intelligent fashion image editing applications.

Xin Chen · 0 citations
Preprint Jul 2026

InnoText: A Unified Model for Visual Text Generation and Editing

Diffusion models have recently achieved remarkable success in high-fidelity image synthesis, yet their application to visual text generation and editing remains relatively underexplored. Unlike general image generation, visual text tasks demand precise structural regularity and legibility, which may pose additional challenges for small-scale text and non-Latin scripts such as Chinese. Existing UNet-based models often struggle to produce clear and coherent text, while DiT-based models, though more expressive, are typically limited to a single task, which may lead to redundant training pipelines, inconsistent visual styles, and reduced cross-task generalization. To address these challenges, we propose InnoText, a unified DiT-based framework capable of performing both text generation and editing within a single model. We introduce a Font Size-Aware Modulation (FSAM) module to enhance representations across font scales, a Small-Character Aware Augmentation strategy to improve fine-grained fidelity, and a Task-Specific Region Weighted Loss for adaptive optimization. To support training and evaluation, we also construct a high-quality bilingual (English-Chinese) visual text dataset covering diverse fonts, sizes, and backgrounds. Experimental results demonstrate that our method achieves superior generation accuracy and editing quality, producing visually appealing and realistic text images.

Haowei Liu, Runze He, Jian Lu et al. · 0 citations