Knowledge reinforced reinforcement learning for transferable selective annotation

Abstract

The increasing data volume in fields like personalized manufacturing, biomanufacturing, and medical diagnostics has amplified the critical role of human annotation, which is time-consuming and labor-intensive, directly impacting AI performance. While active learning aims to reduce annotation efforts by selectively querying data points, existing methods often fail to transfer their sampling strategies across similar-but-not-identical environments. This limitation leads to inefficiencies in data annotation when dealing with varying data distributions or evolving data domains where adaptability is crucial.To overcome these limitations, we propose a knowledge-reinforced reinforcement learning framework inspired by Human Information Processing (HIP) theory. Our hierarchical framework incorporates two cooperative agents: a long-term agent and a short-term agent. The long-term agent, reflecting top-down cognitive processes, strategically guides the learning process by leveraging historical knowledge and overarching objectives, influencing the short-term agent, which focuses on immediate data uncertainties, representing bottom-up processes. By coordinating these agents, the framework enables more effective knowledge transfer and enhances active learning.Our approach enhances the transferability and adaptability of active learning across different but related data domains by facilitating inter-agent cooperation. This hierarchical structure improves sampling efficiency and allows the active learner to generalize its querying strategy to similar-but-not-identical environments. By assigning cooperative goals to each agent, the framework mirrors human cognitive strategies in complex tasks, effectively addressing data annotation challenges in medical and other data-intensive fields, and improving the transferability and effectiveness of AI systems.