Vision-Language Models for Occupational Physical Exposure Assessment: Estimating External Hand Forces in Manual Material Handling Tasks from RGB Video

Quantitative Evaluation of Muscular Risks during Manual Load Handling using Vision-based System and Biomechanical Modeling*

Musculoskeletal disorders (MSDs) continue to represent one of the primary causes of pain, disability, and productivity loss, especially in work environments characterized by repetitive activities and manual material handling. This paper presents a vision-based system to investigate the muscular activities and risks associated with workers performing tasks such as lifting, carrying, and placing a load from a ground pallet to a shelf. A stereocamera setting, utilizing two commercial devices, is used to collect paired images of the actions performed by users. A deep learning model is employed to detect 2D skeletal keypoints, which are then projected into the 3D space using calibration data. Then, the obtained 3D keypoints of the worker’s skeleton enable to derive the biomechanical model of the user by using the OpenSim framework. In this way, it is possible to import the movements of subjects and to perform the force and joint reaction analyses. Different performing metrics are introduced to quantify the duration of muscular activity, the biomechanical efficiency, and the physiological risk associated with different lifting modalities. The results confirm that the proposed approach can provide a first indication of workplace operational configurations that minimize physiological effort and muscular risk for workers.

Markerless Computer-Vision Joint-Angle Analysis for Ergonomic Risk Assessment of Engineering Students During Bench-Work Practicum

Bench work (kerja bangku) is a foundational manual-skills practicum in mechanical and manufacturing engineering education. It requires sustained non-neutral postures—forward trunk flexion, downward neck flexion toward the vice, and repetitive upper-limb exertion—that expose students to work-related musculoskeletal disorder (WMSD) risk early in their careers. Conventional ergonomic evaluation relies on manual observation, which is subjective, labour-intensive, and difficult to scale across large student cohorts. This study presents a markerless computer-vision pipeline that estimates body joint angles from ordinary RGB video and automatically derives Rapid Upper Limb Assessment (RULA) scores for students performing bench-work tasks. Two-dimensional pose estimation localized anatomical landmarks; sagittal joint angles for the neck, trunk, upper arm, lower arm, and wrist were computed from landmark coordinates and mapped to RULA segment scores. Thirty engineering students were recorded performing five representative tasks (filing, hacksawing, marking/scribing, chiselling, and hand-tapping). The mean RULA grand score across tasks was 6.0, with 92% of observations falling in action levels 3–4 (“investigate and change”). Vision-derived joint angles agreed with manual goniometry to within a mean absolute error of 4.8°, and RULA grand scores matched an expert assessor within ±1 point in 96.7% of cases (weighted Cohen’s κ = 0.82). The results show that markerless computer vision offers a low-cost, objective, and scalable instrument for ergonomics education, posture feedback, and bench-station redesign.

An AI-Powered Model for Automatic Real-Time Assessment of Seated Work Postures Using Rapid Upper Limb Assessment (RULA)

Improper seated work postures are common in the workplace and can lead to various musculoskeletal problems, ranging from joint pain to permanent disability. Continuous assessment of seated work postures is necessary to help prevent these risks. However, existing studies have often relied on rule-based methods, which are highly vulnerable to measurement noise from motion capture devices. To overcome this limitation, this study proposes a learning-based approach for posture assessment. Ten participants were recruited to mimic the seven most common awkward and potentially risky seated work postures, while joint angles of their upper body were recorded and computed using both an RGB video-based approach and a Vicon motion capture system-based approach. The RGB measurements were used as features, while the Vicon measurements were used to derive accurate reference labels by comparison against Rapid Upper Limb Assessment (RULA) criteria. A multi-output Random Forest classifier was trained to predict joint-level posture assessment scores, and the model performance was evaluated using a leave-one-subject-out cross-validation scheme. The results provide initial evidence that the model achieved high performance in neck score assessment, while the trunk, shoulder, and elbow scores were more sensitive to posture types and inter-subject differences in posture execution. A larger dataset with more posture types or more subjects would improve the robustness and generalizability of the model.

Human Performance Modeling in Virtual Factories: A Simulation-Driven Ergonomics Approach

Manual workstation operations in manufacturing are traditionally evaluated based on cycle time and task completion, while ergonomic risks and detailed execution behaviors are rarely assessed within the same study. During early workstation planning, this disconnect leaves factors such as sequence deviations, tool use rhythm, shoulder loading, and excessive reach unquantified, hindering the alignment between engineering objectives and human factors requirements. This study proposes a multi view, vision based framework that captures both performance and ergonomic indicators during manual workstation tasks. A precision assembly workstation equipped with three synchronized cameras—one overhead and two lateral cameras—was used to observe three representative tasks: (T1) part picking and placement, (T2) tool-assisted fastening, and (T3) visual inspection. Video data from 18 operators across 324 trials (approximately 10 hours) were processed using RT-DETRv2 for object detection, OCSort for identity tracking, and a pose estimation module for upper-body kinematics. An ROI-based classifier was used to enhance fine-grained component recognition, while multi-view consistency enabled robust event-log generation. The proposed pipeline achieved stable event extraction with an event-level F1 score of approximately 0.88 at near-real-time processing speed. Derived indicators included sequence compliance, cycle time, tool-use rhythm, reach distance, and posture exposure. The results revealed distinct task characteristics: T1 exhibited the highest reach demand, T2 showed the highest shoulder-loading exposure, and T3 involved extended decision-making during inspection. Mixed-effects models confirmed significant task effects on both time-based and posture-related metrics (p < 0.01). Furthermore, a workstation redesign reduced excessive reach by 28%, arm elevation exposure by 19%, and mean cycle time by approximately 9%, demonstrating the value of multi-view vision sensing for ergonomics-informed workstation design.

Biomedical Signal Processing and Control

Markerless On-Device Detection of Compensatory Movement Patterns in Upper-Limb Rehabilitation Exercises from Monocular RGB Video: A Validation Study in Healthy Adults

Neurological disorders drive demand for prolonged upper-limb rehabilitation, yet specialist access is uneven and assessment stays subjective. Marker-based and inertial measurement unit (IMU) systems are accurate but costly and impractical at home, while pose estimation pipelines mostly stop at keypoints, and many process video server-side, raising privacy concerns. We present a markerless pipeline that analyzes monocular RGB video entirely on-device in the browser, so it never leaves the machine. From 33 BlazePose keypoints, it derives five geometric metrics designed to limit body-size dependence—incomplete elbow extension, inter-limb asymmetry, shoulder girdle elevation, lateral trunk lean, and head tilt—compared against empirically calibrated, preliminary thresholds; a finite-state machine segments repetitions, and the flags are pooled into an unvalidated, exploratory quality score. Against an IMU reference over the 0–62∘ range that the recordings cover, the image-plane angle showed a mean absolute error of 2.18∘, below the pre-specified 5∘ tolerance, a trajectory-averaged bias within ±2∘, and Lin’s concordance correlation coefficient of 0.956; the difference is, however, proportional to the angle—about 4% of the measured value—so the accuracy should not be extrapolated to larger elevations, and because that comparison was made offline, it does not include the timing error of the causal real-time path. On a single seated frontal-plane abduction task, with 18 healthy volunteers simulating the compensations and annotated by two independent clinicians blind to the instructed condition, compensation detection reached a macro-averaged F1 of 0.75 and 0.72 against the individual raters. The five signs differ in maturity: near-expert for trunk lean and head tilt, moderate for incomplete elbow extension and inter-limb asymmetry, and weakest for shoulder elevation, which a single frontal view cannot fully disentangle from the abduction motion. Running at 22–30 frames per second on consumer laptops without relying on a discrete GPU, it offers an accessible, privacy-preserving proof-of-concept foundation for home telerehabilitation; generalization beyond this one exercise and effectiveness on genuine post-stroke compensations remain to be established.