The proposed MaskFlow, a training framework for precise localization, consistent background preservation, and seamless boundary transitions, incorporates the mask into the probability path and flow-matching objective, coordinating generation within the editable region with source preservation outside it.
Abstract
Regional image editing has attracted considerable attention for its spatial controllability. Although instruction-based and mask-based editing methods can achieve strong semantic alignment, reliable regional control remains challenging, where an edit must be accurately localized and naturally integrated with the preserved context. We propose MaskFlow, a training framework for precise localization, consistent background preservation, and seamless boundary transitions. MaskFlow incorporates the mask into the probability path and flow-matching objective, coordinating generation within the editable region with source preservation outside it. The proposed Soft-Poisson De-seaming module further refines the predicted vector field during both training and sampling to improve the smooth integration of the edited foreground with the preserved background. We also introduce MaskEdit-Benchmark for general scene and infographics editing, where prompts describe the desired edits without localization cues, leaving masks to specify the target regions. Experiments on natural scenes and infographic images demonstrate consistent improvements over competing methods in both quantitative and qualitative evaluations. Project page: https://reychiaro.github.io/MaskFlow
Text-guided image editing using rectified flow models such as Multimodal Diffusion Transformer (DiT) has demonstrated impressive generation quality. However, existing methods apply edits globally, inevitably modifying background regions unrelated to the intended semantic change. Our key observation is that the Text-to-...
Trong-Tai Dam Vu, Vinh-Tiep Nguyen· International Conference on...· 0 citations
SR-Edit is proposed, an image editing framework that overcomes issues via iterative self-refinement and achieves superior preservation and overall image quality compared to existing editing techniques.
Andong Wang, Zehua Chen, Yuxuan Jiang et al.· 0 citations
This work revisits one-step image editing from a spatially controlled perspective and proposes WhereEdit, a framework that reformulates one-step editing as localized adaptive editing that consistently outperforms existing one-step image editing methods, achieving superior editing quality while maintaining the efficienc...
Ming Hu, Ming-Yu Dou, Jian-Fu Yin et al.· arXiv.org· 1 citation
This work implements overpainting by adapting a pretrained image editing diffusion model using a combination of joint attention and low-rank adaption across input images with attention-dropout to balance the information flow between noise, source and mask images.
Sam Sartor, Iliyan Georgiev, Michael Fischer et al.· 2 citations
Controllable local editing of 3D assets requires precise target localization and appropriate visual guidance. However, existing methods lack a simple yet accurate way to obtain 3D masks and struggle to achieve the desired edit while faithfully preserving the structure and appearance of non-target regions. To address th...
R. Nie, Chuang Wang, Haitao Zhou et al.· 0 citations
This work proposes EditBridge, a diffusion bridge framework for efficient ultra high-resolution editing that achieves high-fidelity editing with superior perceptual quality at resolutions up to 4K, delivering 3.6--8.4$\times$ speedup at 2K and enabling practical 4K editing in 61 seconds.
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.