FenceXR: AR Movement Replay for Error-Detection Training and Spatially Grounded Feedback

Abstract

Recognizing technical errors in movement is a perceptual skill important to motor learning, but it is challenging for beginners in fast, complex sports like fencing to develop it. A coach's attention is scarce, live demonstrations vary from repetition to repetition, and video review is limited to whatever camera angle was used to record it. Coaches and advanced fencers reviewing a recording face a related problem. They can see an error, but have no way to anchor their feedback to the movement itself, and are left describing it in words the learner must map back onto their own body. We present FenceXR, an augmented reality system that reconstructs 3D movement replays from monocular smartphone video to address both problems. A Trainee module trains novices to detect common lunge errors while a Reviewer module lets coaches and advanced fencers attach text or voice annotations to a specific joint and moment in a replay, which can be shared asynchronously with a trainee. In a study with 18 novice fencers, unaided error-detection accuracy rose from near-chance (37.5%) before training to 64.1% after a single session, with interviews showing a shift from broad visual scanning toward targeted inspection of specific joints and their timing. In a video-based study with four fencing experts, all four viewed the Reviewer module as a valuable complement to their existing coaching tools, particularly for feedback that is difficult to convey through standard video. We end with a discussion of implications for designing AR systems that ground movement-based training and feedback in the movement itself.