Knowledge- and memory-based interactive task learning for human robot collaboration
Abstract
This work proposes a novel framework for interactive task learning that combines essential task knowledge with experiential tuning, a combination we need for cognition-inspired learning for robots. Interactive task learning is an approach inspired by research on cognitive development that promotes story-based memory structures for learning the essence of a task, which are extended and optimized over time by experiences. The proposed framework combines two recent approaches: (1) deploying actionable knowledge graphs (AKGs) as a knowledge base to help robots understand and generalize tasks and (2) integrating Narrative-Enabled Episodic Memories (NEEMs) as semantic and explainable memory structures for experiences. We integrate AKGs into the NEEM structure to benefit from the actionable function description as part of a structure that improves by ongoing and consecutive interactions. If robots grasp the essence of a task, they can transfer knowledge to similar ones. For example, they can use the knowledge of “cutting bread” to “slicing a cucumber,” distinguishing between similar tasks like cutting and slicing, but also reasoning about the similarity of needed tools and motions. Our framework links the existing concepts AKGs and NEEMs in a scenario of task demonstrations for robots to reason about semantic task and object information. By inferring task parameters like cut positions, pour angles, and stop conditions directly from AKGs, robots can achieve task generalization across diverse actions. Our example study setup for the proposed model illustrates this for household tasks, focusing on fruit-cutting and pouring. This is a hypothesis paper: we present the framework and two planned study designs, and we do not report robot experiments or task-success measurements. An example NEEM can be accessed and queried on the resource website 1 .