Security and Safety of Large Language Models—A Use Case for Extended Reality Environments
TL;DR
Recommendations for the design and deployment of secure LLM-enabled XR systems, emphasizing strategies such as security defense in depth, privacy-aware data practices, layered technical safeguards, continuous monitoring, and governance mechanisms that align security controls with the specific risks of immersive and interactive environments are proposed.
Abstract
This research investigates the security of large language models (LLMs) with the aim of identifying key security and safety threats, control measures, and governance considerations that are relevant to their trustworthy adoption in the Extended Reality (XR) domain. To achieve this goal, the research combines an extensive narrative literature review with a workshop with 21 participants who have an AI background and are early-stage technical professionals. The workshop was designed to capture informed perspectives and critical reflections on this behalf. The findings show that LLMs security is understood as a lifecycle-wide, socio-technical challenge in which data quality, privacy, model robustness, adversarial resilience, human misuse, monitoring, and regulatory compliance are deeply interconnected. Building on these insights, this research proposes a set of recommendations for the design and deployment of secure LLM-enabled XR systems, emphasizing strategies such as security defense in depth, privacy-aware data practices, layered technical safeguards, continuous monitoring, and governance mechanisms that align security controls with the specific risks of immersive and interactive environments.