General‐purpose agents in human‐machine teams: Can we preserve meaningful human control?

Abstract

As General‐Purpose AI (GPAI) grows more capable of cross‐domain reasoning and multi‐agent coordination, it reshapes the design space of military human‐machine teams and introduces new opportunities for adaptability as well as new risks for Meaningful Human Control (MHC). The paper presents an integrated delegation framework for preserving MHC in GPAI‐enabled human‐machine teams. It combines play‐based task structuring, normative constraints, LLM‐mediated interaction, and an exploratory empirical evaluation with military professionals. Part of this framework is DIALOG, an LLM‐mediated interaction component that translates natural language into doctrine‐aligned plays, and builds up a context with ethical, legal, and operational constraints. We evaluate this framework in TNO's vehicle simulation facility, where five military professionals executed tactical missions using closed play, open play, and goal‐based delegation modes. Findings reveal a nuanced autonomy‐control trade‐off: operators initially favored higher‐autonomy modes but reverted to more structured control when confronted with unexpected or non‐doctrinal agent behaviors. The results highlight implications for MHC, transparency, and trust calibration, and point toward research directions for enhancing tactical competence, communication fidelity, and verifiability in GPAI‐enabled human‐machine teams.