Robust Exploration in Directed Controller Synthesis via Mixture-of-Experts Reinforcement Learning

Abstract

Controller synthesis is a promising approach as a planner for self-adaptive systems, as it can automatically re-synthesize control strategies that satisfy the specified properties in response to runtime changes. To enhance efficiency, Directed Controller Synthesis prunes the search space by incrementally constructing a partial view of the system, aiming to find a valid controller without exhaustive exploration. This process is steered by an exploration policy (i.e., heuristic), and Reinforcement Learning has proven highly effective for learning such policies. However, a key challenge is anisotropic generalization, i.e., a policy trained on specific domain parameters is specialized, performing well in certain scenarios while remaining fragile in others. To this end, we propose a Mixture-of-Experts framework that combines multiple policies, leveraging their complementary strengths to form a more robust exploration policy. The evaluation on the Air Traffic benchmark shows that our proposal significantly increases the number of solvable instances.