Mixed reality systems for space exploration integrating telemetry data, artificial intelligence, three dimensional modeling, and hardware interfaces

Abstract

This paper presents a comprehensive review of how Mixed Reality (MR) systems can enhance space mission operations by combining real-time telemetry data, Computer-Aided Design (CAD)-based Three-Dimensional (3D) modeling, and Artificial Intelligence (AI).These technologies offer immersive platforms for decision-making, anomaly detection, and predictive analysis of space missions for example, through Long Short-Term Memory (LSTM) networks for time-series predictive maintenance. MR platforms provide immersive and interactive environments that enhances situational awareness, anomaly detection, and decision-making across different phases of space missions. MR hardware, such as Microsoft HoloLens and Varjo XR-4, enables astronauts and ground crews to collaborate with virtual spacecraft models superimposed with telemetry data, enabling real-time troubleshooting and collaborative decision-making. The review paper further investigates Artificial Intelligence and Machine Learning (AI/ML) integration in context-aware guidance, highlighting the importance of edge-cloud architectures formalized as two-tier Decision Support System (DSS) for achieving low latency constraints, system integration complexity, and secure collaboration. The review also examines the use of virtual modeling software to create detailed 3D spacecraft models for simulation and mission planning, thereby improving operational precision and risk assessment. Challenges and existing limitations are discussed along with proposed solutions. Finally, the paper highlights emerging trends in MR hardware and AI innovation, outlining their potential as promising developments that could enable new functionalities in future missions, by improving space operations through greater safety, efficiency, and collaboration. The transformative potential of MR is highlighted throughout this review at all stages of the space operations timeline, including pre-launch and launch planning, in-mission execution, and astronaut training.