Machine failures, which represent forms of performance degradation, are common in real-world manufacturing systems, however, they are often overlooked in job shop scheduling solutions that primarily focus on complete machine breakdowns. These subtle disruptions can lead to cascading delays and reduced system efficiency. This study proposes a reinforcement learning (RL) framework designed to address the Job Shop Scheduling Problem (JSSP) in environments affected by such failures. Unlike traditional RL-based scheduling models that concentrate on total breakdowns, this work considers more nuanced disruptions, such as processing slowdowns, which frequently occur in practical settings. The proposed framework enhances a Q-learning algorithm by introducing a refined environment that incorporates an extended state representation and a reward function tailored to account for performance degradation. These enhancements enable the RL agent to learn adaptive scheduling policies that minimize makespan while effectively responding to partial machine failures. The framework is validated using the Taillard benchmark dataset across varying levels of disruption and job-machine configurations. Experimental results show that the proposed environment consistently delivers superior scheduling performance compared to baseline models that do not consider machine failures. The findings highlight the framework’s potential to improve scheduling resilience and efficiency in dynamic production environments.
PGMPO is proposed, a novel learning framework consisting of a simple but effective multi-policy modeling approach that allows a single network to represent multiple decision-makers, and a preference-driven model optimization method that effectively guides policies to learn diverse and specialized problem-solving strategies without the need for explicit reward functions.
Inguk Choi, Woo-Jin Shin, Sang-Hyun Cho et al.· 0 citations
Job scheduling in Electronic Design Automation (EDA) environments presents unique challenges due to high-frequency job submissions, short job durations, and strict latency requirements. Production schedulers such as IBM Spectrum LSF employ robust heuristics like First-Come-First-Served (FCFS) that provide predictable behavior and fairness guarantees. However, these rule-based approaches do not learn from historical workload patterns, leaving potential for further optimization through adaptive methods. We present Adaptive Job Selection (AJS), a reinforcement learning-based agent that learns to schedule pending jobs for reduced job waiting and completion time on LSF clusters for EDA workloads. AJS introduces macro actions that dispatch multiple jobs per inference, addressing the credit assignment problem inherent in high-frequency scheduling while meeting real-time latency constraints. Our lightweight neural network architecture employs cross-attention to capture interactions between job buckets and cluster state, enabling inference at the frequency required by EDA workloads. We deploy AJS as an external plugin in IBM Spectrum LSF and evaluate it on an IBM LSF cluster. Experiments demonstrate that AJS achieves a 62.6% reduction in average job waiting time, a 19.8% reduction in job completion time, a 16.0% improvement in job throughput, and 3.05 percentage points higher CPU utilization compared to the default scheduler. We also share practical lessons for bridging the simulation-to-production gap. To our knowledge, AJS is the first open-source, deployable RL-based scheduler designed for production EDA environments.
Yiming Shao, Aijun An, Michael Spriggs et al.· Proceedings of the 32nd ACM...· 0 citations
Modern manufacturing requires scheduling methods that adapt to changing order arrivals, machine disruptions, customer priorities, stakeholder preferences, and time-varying energy conditions. This paper proposes a preference-conditioned deep reinforcement learning (DRL) approach for dynamic scheduling in sustainable and robust manufacturing. The approach is embedded in a cyber-physical production system (CPPS)-oriented framework that links production states, machine availability, energy-related background data, simulation-based learning, performance monitoring, and decision support. Within this framework, a Double Deep Q-Network (DDQN) scheduler is developed for joint job sequencing, machine assignment, and start-time adjustment. The scheduler uses a candidate-based state representation for dynamic order arrivals, vector-valued Q-output for objective-specific value estimation, and a priority- and preference-aware reward design. Customer priorities are treated as order-level attributes, while stakeholder preferences are encoded as system-level objective weightings. This enables one policy to consider energy-related cost, carbon emissions, energy demand, and tardiness while adapting to different preference profiles. The concept is demonstrated in an on-demand manufacturing (ODM)-oriented parallel CNC machining case with heterogeneous orders, product-specific setup and processing requirements, hourly electricity prices, carbon-intensity signals, and curriculum-adaptive machine breakdowns. DDQN is compared with three dispatching rules and two DRL baselines under shared training and testing scenarios. The results show that DDQN achieves the lowest energy-related cost and carbon emissions in training and unseen testing while maintaining acceptable delivery performance. Overall, the study demonstrates the potential of CPPS-oriented and preference-conditioned DRL for adaptive, energy-aware, and robust scheduling in smart manufacturing systems.
Chao Zhang, Gabriela Ventura Silva, Christoph Herrmann· Production Engineering· 0 citations
This paper addresses the Job-Shop Scheduling Problem in dynamic semiconductor manufacturing environments by proposing a decentralized, bottom-up Artificial Bee Colony (ABC) scheduling algorithm. Machines and lots are modeled as autonomous agents whose local interactions give rise to system-level scheduling behavior. Alongside classical scheduling objectives, the proposed approach focuses on regulating production dynamics by maintaining sufficient diversity in machine queues, formalized through entropy-based measures. Bottlenecks are treated not only as a consequence of static capacity constraints relative to work in progress, but also as emergent effects of short-term demand concentration, where multiple lots converge toward the same resources within limited time horizons. To manage these effects, a fitness formulation is introduced that promotes balanced queue states through local decision-making. Scheduling foresight is incorporated via a Look-Ahead Window, while uncertainty in distant future routing is accounted for using a decay factor, jointly enabling adaptive prioritization under bounded computational effort. Simulation-based evaluation across fabrication scenarios of increasing scale shows that the method achieves modest improvements in Flow Factor and Tardiness, while inducing an expected trade-off in Makespan under higher load conditions. More importantly, analysis of queue-level dynamics reveals more regular behavior in the evaluated scenarios, with reduced fluctuations in queue lengths, batch waiting, and minimum queue entropy over time. These results indicate that the proposed ABC-based approach can improve observed predictability at the queue level, offering a complementary perspective to performance-driven scheduling in highly dynamic environments.
Rising energy costs and the increasing share of renewable generation create incentives to align production schedules with dynamic electricity prices and on-site solar generation. We address the resulting Multi-Objective Flexible Job Shop Scheduling Problem by proposing a deep reinforcement learning framework that jointly minimizes makespan and energy cost. A single preference-conditioned policy approximates the Pareto front at inference time, eliminating the need to train separate models for different objective weightings. The agent acts as a hyper-heuristic, selecting among heuristic actions at each decision point, including strategies that intentionally delay operations to exploit periods of lower electricity prices or higher solar generation. Preferences are integrated throughout the network via Feature-wise Linear Modulation, while a dual-critic architecture and a diversity loss preserve preference-specific policy behaviors. We evaluate the approach against NSGA-II and Joined Heuristics on synthetic instances ranging from 10×5×5 to 15×15×15 jobs, operations per job, and machines using normalized hypervolume and inverted generational distance. While NSGA-II performs best on the smallest instances, the proposed approach becomes increasingly competitive as problem size grows and achieves the best results on the largest evaluated instances. These findings indicate promising scalability within the investigated problem range.
Dustin Moreira Simoes, Marvin Brune, Mehmet Ulrich et al.· Applied Sciences· 0 citations
When flexible job shop scheduling faces concurrent disturbances such as machine failures and rush orders, worker-centric constraints emphasized under Industry 5.0 must also be satisfied. Existing deep reinforcement learning methods for the Dynamic Flexible Job Shop Scheduling Problem (DFJSP) seldom treat worker workload balance as an explicit constraint, and most depend on static penalty coefficients that are difficult to tune across different scenarios. In this paper, we suggest ALP-PPO, an adaptive Lagrangian penalty-enhanced proximal policy optimization algorithm, for real-time rescheduling under concurrent machine breakdowns and rush orders. We formulate the scheduling environment as a constrained Markov decision process. Worker skill heterogeneity, fatigue accumulation and workload equity are modeled as coupled constraints alongside classical scheduling objectives. By decoupling operation sequencing, machine allocation and worker assignment into coordinated sub-decisions, a hierarchical action space is constructed. Dual Lagrangian multipliers for workload balance and fatigue are updated adaptively during training, so that manual penalty tuning is no longer required. An event-triggered mechanism selects between right-shift and full rescheduling on the basis of a disruption severity index. We employ weighted-sum scalarization of makespan, energy consumption and workload variance during training, and Pareto solution sets are obtained by systematically varying the weight vectors across independent training runs. On extended Brandimarte benchmarks augmented with worker and dynamic event parameters, ALP-PPO delivers superior scheduling performance across makespan, energy consumption and workload variance when compared with Double DQN, Dueling DQN, standard PPO, NSGA-II and MOEA/D, as measured by Hypervolume (HV) and Inverted Generational Distance (IGD) indicators. Ablation studies indicate that the adaptive Lagrangian mechanism reduces constraint violations by more than 40% relative to fixed-penalty alternatives while keeping the primary objectives competitive. An analysis of computational efficiency shows that ALP-PPO completes online inference in under 20 ms per decision step, making real-time rescheduling practically feasible. Generalization experiments on previously unseen instances further validate the transferability of the learned policy. These findings support human-centric intelligent scheduling in Industry 5.0 manufacturing.