Preference-guided graph reinforcement learning for multi-objective quality-aware hybrid flow shop scheduling with multilevel assembly

Abstract

Product quality is crucial to the competitiveness of manufacturing enterprises. However, existing quality-aware scheduling studies mainly approximate product quality evolution using indirect factors such as processing time, making it difficult to characterize the integrated effects of equipment health conditions and operation precedence relationships on product quality. To address this issue, a quality-aware hybrid flow shop scheduling problem with multilevel assembly (QHFSP-MA) is formulated by incorporating multilevel assembly processes and processing-assembly quality propagation mechanisms of complex precision products. Due to the large solution space and multi-objective characteristics of QHFSP-MA, existing multi-objective reinforcement learning methods still struggle to obtain high-quality Pareto fronts. Therefore, a preference-guided graph reinforcement learning framework (PGRL) is proposed. A heterogeneous graph is first constructed to represent the processing-assembly system. Then, a preference-guided meta-relation heterogeneous graph neural network is developed to dynamically adjust the importance of different meta-relations according to objective preferences, enabling generalized policy learning under diverse preferences. Furthermore, an adaptive preference weight mechanism (APWM) is designed to reduce the mismatch between uniform preference sampling and irregular Pareto front distributions, thereby improving Pareto front approximation. Experimental results demonstrate that PGRL outperforms dispatching rules, metaheuristic algorithms, and two advanced multi-objective deep reinforcement learning methods in terms of optimization performance, generalization, and computational efficiency. An automotive engine manufacturing case further validates its practical effectiveness, reducing makespan, quality loss cost, and maintenance cost by 27.3%, 25.7%, and 8.8%, respectively.