Multiscale context aggregation and residual fusion for robotic grasp detection

Abstract

Robotic grasp detection in unstructured environments demands accurate multi-scale context representation and semantically consistent feature fusion across decoder stages. Existing encoder–decoder approaches either aggregate context at a single receptive field scale or fuse skip connections through direct element-wise addition, both of which limit detection accuracy for objects of diverse sizes and orientations. In this paper, we propose MSCARFNet, an end-to-end RGB-D grasp detection network built around two novel components. The Multi-Scale Context Aggregation (MSCA) module enriches the bottleneck representation by aggregating features from four parallel branches—a 1 × 1 convolution and three 3 × 3 dilated convolutions with rates {2,4,6}—and adaptively re-weights them via a channel-wise SE gate, enabling the network to jointly capture fine, medium, and coarse contextual cues in a single unified module. The Residual Fusion Block (RFB) replaces naive skip-connection addition in the decoder with a residual-refined fusion scheme that projects deep and shallow features to a common channel space, sums them, and applies residual refinement to suppress semantic mismatch while preserving spatial detail. Extensive experiments on the Cornell Grasping Dataset and the Jacquard dataset demonstrate that MSCARFNet achieves 99.1% / 98.2% accuracy (image-wise / object-wise split) on Cornell and 95.4% on Jacquard at an inference speed of 21 ms per frame. Comprehensive ablation studies confirm the individual contribution of each component as well as the importance of the asymmetric dilation configuration and the residual refinement design within each module.