Measuring Situation Awareness During Teleoperation: Scenarios, Sensitivity, and Predictiveness of Operator Performance

Abstract

Acquiring and maintaining situation awareness (SA) is central for human operators during teleoperation. Therefore, examinations of interface design approaches or conditions that facilitate adequate operator SA are frequent in teleoperation research. Such investigations typically require the measurement of operator SA, for which various approaches are distinguished. These include direct objective approaches that pose questions regarding the situation to operators, direct subjective approaches that rely on self-ratings, and indirect approaches that infer SA from performance or related processes. The objective of the current research was to provide an initial investigation of how operator SA is measured during teleoperation by accounting for typical measurement scenarios. Based on a systematic literature review, measures from all three approaches were identified in teleoperation research that assessed operator SA as a dependent variable. Experimental variations in the identified studies typically referred to (a) the operator displays, (b) the control of the telerobot, and (c) task characteristics. The sensitivity and predictiveness of SA measures were examined based on a meta-analysis. Reduced sensitivity was identified for experimental variations reflecting telerobot control. The predictiveness of SA measures for teleoperation performance was characterized by a medium positive correlation across four studies, conforming with quantitative research syntheses that included other domains. Overall, the results indicated a potential lack of suitable SA measures for the evaluation of systems that vary aspects related to the control of the telerobot. Recommendations for researchers seeking to select an existing measure of operator SA for evaluating teleoperation systems are provided.