TriSelect: Distance Adaptive Multimodal Object Selection in Virtual Reality Using Eye Gaze, Hand Gesture, and Voice

Abstract

Object selection in virtual reality (VR) is a deceptively difficult problem: the same user must reach for a pen on their desk, point to a chart across the room, and answer a question on a distant whiteboard yet no single input modality performs well across all three. Gaze suffers from unintentional activation, hand gestures become physically impractical at range, and voice introduces latency and noise sensitivity. We present TriSelect, a distance-adaptive multimodal selection system that addresses this by dynamically weighting three input modalities — eye gaze, hand pinch gesture, and voice command — based on the user’s spatial relationship to the target. TriSelect partitions the interaction space into three zones (Near: 0–1.5m, Mid: 1.5–4m, Far: 4m+) and applies a FusionEngine that shifts modality weights accordingly. Selection is confirmed when a combined weighted confidence score exceeds 0.55 and at least two modalities agree within a 1.2-second temporal window. Implemented on the Meta Quest 2 using Unity and OpenXR, TriSelect is presented as an early-stage system contribution demonstrated in a VR classroom scenario spanning all three zones; formal user evaluation is deferred to future work. To our knowledge, this is the first system to fuse all three modalities with explicit distance-adaptive zone weighting, contributing a novel interaction paradigm for context-aware multimodal VR interfaces.