Disturbance-Augmented Simulations for Improved Sim-to-Real Transfer of Compliant Reinforcement Learning-Based Robotic Control

Abstract

Sim-to-real transfer enables scalable acquisition of compliant robotic skills for contact-rich tasks, yet dynamics mismatches between simulation and hardware hinder reliable deployment. Prior approaches assume disturbance-free training dynamics, neglecting endogenous disturbances from nominalplant modelling errors that degrade real-world performance. This simplification limits transfer of high-compliance six-DoF skills, as prior successes relied on lower-compliance or three-DoF control. Consequently, current methods cannot support skill acquisition for high-dexterity tasks demanding high compliance. We address this gap with Disturbance Augmented Simulations for improved sim-to-real TRAnsfer (DASTRA), a general simulation design framework for structured disturbance modelling to produce disturbance-aware policies more robust to real-world modelling errors. Guided by DASTRA, we propose four disturbance models-inertial parameter mismatch, friction extensions, joint sensor bias, and policy delays-and evaluate their effect on sim-to-real transfer of a six-DoF peginsertion skill. Results identify inertial mismatch, sensor bias, and policy delays as primary drivers of transfer robustness, providing immediately usable simulation enhancements for high-compliance manipulation tasks. Enhanced friction modelling did not improve performance, underscoring the need for careful disturbance model design. Collectively, these findings establish disturbance-augmented simulator design as essential for reliable transfer of high-dexterity, high-compliance skills. Beyond peg insertion, DASTRA can enable scalable deployment of compliant skills in challenging industrial settings such as assembly and disassembly, and motivates continued refinement to further close the sim-to-real dynamics gap.