Human Performance Modeling in Virtual Factories: A Simulation-Driven Ergonomics Approach

Abstract

Manual workstation operations in manufacturing are traditionally evaluated based on cycle time and task completion, while ergonomic risks and detailed execution behaviors are rarely assessed within the same study. During early workstation planning, this disconnect leaves factors such as sequence deviations, tool use rhythm, shoulder loading, and excessive reach unquantified, hindering the alignment between engineering objectives and human factors requirements. This study proposes a multi view, vision based framework that captures both performance and ergonomic indicators during manual workstation tasks. A precision assembly workstation equipped with three synchronized cameras—one overhead and two lateral cameras—was used to observe three representative tasks: (T1) part picking and placement, (T2) tool-assisted fastening, and (T3) visual inspection. Video data from 18 operators across 324 trials (approximately 10 hours) were processed using RT-DETRv2 for object detection, OCSort for identity tracking, and a pose estimation module for upper-body kinematics. An ROI-based classifier was used to enhance fine-grained component recognition, while multi-view consistency enabled robust event-log generation. The proposed pipeline achieved stable event extraction with an event-level F1 score of approximately 0.88 at near-real-time processing speed. Derived indicators included sequence compliance, cycle time, tool-use rhythm, reach distance, and posture exposure. The results revealed distinct task characteristics: T1 exhibited the highest reach demand, T2 showed the highest shoulder-loading exposure, and T3 involved extended decision-making during inspection. Mixed-effects models confirmed significant task effects on both time-based and posture-related metrics (p < 0.01). Furthermore, a workstation redesign reduced excessive reach by 28%, arm elevation exposure by 19%, and mean cycle time by approximately 9%, demonstrating the value of multi-view vision sensing for ergonomics-informed workstation design.