Mid-air Hand Gesture Sequences: Evaluation and Design Recommendations Towards Gestural-Interfaces

Abstract

Mid-air hand gestures enable users to execute commands in on-the-go contexts, especially when other input methods are unavailable. However, in order to complete complex tasks, users must string gestures together into a sequence; this paper breaks down the unique design considerations and human factor issues introduced when performing mid-air hand gesture sequences. A first study provides empirical evidence supporting their viability and identifying unique characteristics versus singular gestures. A second study compares gesture sequences to mouse- and touch-based input in a cafe menu ordering scenario. The evidence suggests that gesture sequences can play an important role in the landscape of input methods by maintaining the benefits of gestures while enabling users to accomplish complex tasks. Based on these two studies, we synthesized design recommendations such as designing vocabularies that avoid intermediate gestures and maintaining a balance of similar gestures to improve fluidity without sacrificing discriminability.