Skip to content
#diffusion models Open access

Robotic Hand‐Arm Coordinated Skill Learning for Tool Manipulation Tasks

Oct 2026 · CAAI Transactions on Intelligence Technology · 0 citations · 17 references

Abstract

This paper focuses on the specific scenario of tool manipulation and presents a novel framework enabling robots to acquire tool manipulation skills through learning from human demonstrations. While there have been numerous studies on robotic manipulation skill learning, most of them fail to fully leverage the combined advantages of coordinated hand‐arm movements—such as in dynamic tool repositioning and complementary workspace utilisation. To tackle this issue, the framework presented in this paper captures visual and tactile sensory streams from human demonstrations—including synchronous camera images and tool interaction forces acquired through optical tactile sensing—while recording hand‐arm coordination strategies. A Diffusion Policy model is adopted to learn robust manipulation skills from the demonstration data. During task execution, the robot fuses real‐time visual and tactile feedback to dynamically coordinate hand and arm motions. This work innovatively introduces object‐level stiffness as a feature characterising hand manipulation actions, allowing the robot to exhibit human‐like adaptive capabilities during skill deployment. We validate the proposed method in two real‐world tasks: cucumber peeling and bottle squeezing. Results show the robot achieves reliable manipulation under complex constraints and unknown perturbations. Multimodal imitation learning with fused vision‐tactile feedback enables efficient hand‐arm coordination, improving the robustness and flexibility of robotic tool manipulation.

Read PDF

Similar papers

#computer vision Open access Jun 2016

Software Development in Startup Companies: The Greenfield Startup Model

The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.

Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al. · 178 citations · ⚡14
#computer vision Open access Oct 2016

Software Startups - A Research Agenda

Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.

M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al. · 157 citations · ⚡17
#machine learning Review Open access Oct 2016

“Failures” to be celebrated: an analysis of major pivots of software startups

This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.

Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al. · 127 citations · ⚡15
#computer vision Review Open access May 2015

A survey study on major technical barriers affecting the decision to adopt cloud services

The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.

Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al. · 111 citations · ⚡8
#large language models Review Open access Nov 2020

Status of the WHO recommended timing and frequency of antenatal care visits in Northern Bangladesh

Objective There is dearth of information on the timeliness of antenatal care (ANC) uptake. This study aimed to determine the timely ANC uptake by a medically trained provider (MTP) as per the World Health Organization (WHO) recommendations and the country guideline. Methods Cross-sectional survey was done with 2,731 wo...

B. Sarker, Musfikur Rahman, T. Rahman et al. · 89 citations · ⚡7
#computer vision Open access Feb 2018

Lean Internal Startups for Software Product Innovation in Large Companies: Enablers and Inhibitors

This study investigates how Lean internal startup facilitates software product innovation in large companies and identifies its enablers and inhibitors, and shows the potential of the method-in-action framework to investigate the Lean startup approach in non-startup context.

Henry Edison, Nina M. Smørsgård, Xiaofeng Wang et al. · 78 citations · ⚡6

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.