Aug 2026· International Conference on Image Processing. Machine Learning and Pattern Recognition· Vol 14304, pp. 1430404 - 1430404-9· 0 citations· 12 references
Engineering
TL;DR
The Regional Self-Attention Mechanism is proposed, which strictly confines the self-attention operation to the independent regions defined by the target bounding boxes, enabling the model to generate specific-category targets in a targeted manner during the denoising process.
Abstract
Layout-to-image diffusion models have demonstrated significant potential in generating multi-object, position-controllable images. However, this paper identifies three fundamental contradictions in existing methods: difficulty in distinguishing overlapping objects of the same type, background-dominated training leading to the suppression of small objects, and semantic misalignment between generated images and layout annotations. By analyzing the self-attention mechanism in diffusion models, we identify its global receptive field as the root cause of these issues—when the bounding boxes of multiple objects of the same class overlap spatially, the model lacks a mechanism to ensure that pixels within each region focus exclusively on the corresponding object. To address this, we propose the Regional Self-Attention Mechanism, which strictly confines the self-attention operation to the independent regions defined by the target bounding boxes, enabling the model to generate specific-category targets in a targeted manner during the denoising process. Experiments on datasets such as COCO and Visual Genome demonstrate that this method significantly improves the distinguishability of overlapping objects of the same class, the reconstruction accuracy of small objects, and the semantic consistency between layout and image.
Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images. Diffusion models are promising for this task because of their strong generative performance. However, existing diffusion-based methods typically use semantic priors only as external conditions, without explicitly regulating token interactions within the denoising network. Consequently, they struggle to preserve object locations, shapes, and semantic layouts required for reliable annotation reuse. We propose SC-Diff, a semantically calibrated latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration. A pretrained SAM3 model with predefined text prompts first extracts category-specific semantic masks from visible images. These masks are merged into a semantic map and fused with the visible image as the input condition. The same map is converted into token-level semantic labels to calibrate self-attention in the denoising network. Based on these labels, we introduce Semantic-Guided Self-Attention Calibration (SGSC), which adaptively applies positive biases to query-key pairs of the same category. The query-wise calibration strength depends on the dispersion of attention across semantic categories and the attention assigned to the query's own category. The original attention scores further modulate the bias, giving greater calibration to same-category keys with stronger responses. This soft calibration reduces cross-category interference while retaining global contextual interactions, thereby improving semantic consistency in generated infrared images. Extensive experiments show that SC-Diff improves perceptual quality and produces more effective synthetic training data for downstream infrared object detection.
Junyin Zhang, Siyu Huang, Jianxiong Ye et al.· 0 citations
Layout-to-image diffusion models have achieved impressive semantic controllability by conditioning generation on category-level segmentation maps. However, such category-aligned control is not necessarily instance-addressable: multiple nearby objects from the same category are often treated as a shared semantic region, leading to ambiguous boundaries, averaged appearances, and feature confusion among instances. This limitation is particularly evident in urban scene synthesis, where small and crowded pedestrians or vehicles require fine-grained instance separation while preserving global scene consistency. In this paper, we propose InstancePin, an instance-addressable layout-to-image diffusion framework that pins each object instance with an explicit coordinate anchor. Instead of directly injecting instance masks into the pretrained backbone, InstancePin introduces an independent instance-aware adapter to preserve the category-level generation prior while learning instance-specific spatial control. For each instance, its center coordinate is encoded with Fourier features and projected into a coordinate token, which serves as a spatial anchor queried by latent image features through coordinate pinning attention. To make these anchors spatially meaningful, we further supervise the coordinate attention maps with instance regions, encouraging each coordinate token to activate its corresponding object area. Finally, an instance-mask guided fusion module routes pretrained backbone features to non-instance regions and adapter features to instance regions, enabling local instance refinement without sacrificing global semantic fidelity. Extensive experiments on Cityscapes demonstrate that InstancePin mitigates instance entanglement in dense layouts and improves both image fidelity and semantic consistency.
To address the identification and segmentation challenges caused by large bubble size variations, dense distributions, and severe overlap and occlusion in gas–liquid two-phase flows, a global-local fusion segmentation model named global-local bubble network is designed. The model uses the visual state space encoder as the backbone for feature extraction. By utilizing the two-dimensional selective scan mechanism with linear complexity, it establishes long-range pixel dependencies, significantly enhancing context inference capabilities under dense occlusion. Simultaneously, a global-local slicing interface is proposed, which effectively eliminates edge truncation artifacts inherent in high-resolution image tiling through overlap-based slicing augmentation during training and boundary-aware reconstruction inference during inference, ensuring the topological integrity of cross-scale targets. Finally, an adaptive receptive field aggregation neck incorporating a selective Kernel Atrous spatial pyramid pooling module is introduced to dynamically adjust receptive field weights and fuse deep semantic features with shallow texture features, thereby effectively improving the segmentation accuracy of multi-scale bubbles. Compared with mainstream single-stage and multi-stage instance segmentation frameworks, the proposed model shows superior segmentation performance and robustness in simulation experiments. Its comprehensive F1-score reaches 99.89%, and the Mean Intersection over Union (mIoU) reaches 95.11%. Applied to real bubbly flow experiments, it attains an average F1-score of 91.55% and an average mIoU of 94.39% under rigorous fivefold cross-validation, maintaining superior and stable performance under complex conditions. This provides a reliable solution for the refined parameter statistics of complex multiphase flow fields.
Mi Wang, Kaijie Tong, Xinxin Li et al.· The Physics of Fluids· 0 citations
Salient Object Detection (SOD) remains a fundamental task in computer vision and visual computing, supporting applications ranging from image understanding to human-computer interaction. Existing methods still face two coupled challenges: insufficient modeling of multi-scale salient structures and imbalanced fusion between global semantic information and local details, which often lead to incomplete salient regions and blurred boundaries. To address these issues, this study proposes MSGAN, a multi-scale global-local collaborative learning framework that integrates multi-scale mixed convolution and adaptive global-local attention to enhance feature representation. Extensive experiments on the HKU-IS, ECSSD, PASCAL-S, and DUT-OMRON datasets demonstrate that our method achieves significant improvements in F-measure, MAE, and Em metrics, outperforming state-of-the-art approaches. Ablation studies validate the effectiveness of each core component. This work advances robust SOD for complex real-world scenarios and provides insights into attention-guided visual perception.
