Adaptive CPU Scheduling using a Q-Learning Meta-Scheduler

Abstract

Conventional CPU scheduling policies such as FirstCome First-Served (FCFS), Shortest Job First (SJF), Round Robin(RR), and Priority scheduling apply fixed, pre-defined rules thatare optimal only under specific workload conditions and cannotadapt to dynamic and heterogeneous execution environments. Thispaper presents an adaptive meta-scheduler that employs tabularQ-learning to dynamically select among classical schedulingpolicies based on the runtime state of the system. A discreteevent CPU simulation environment models process lifecyclebehaviour, and a Gymnasium-compatible reinforcement learning(RL) environment exposes a discretised five-dimensional staterepresentation, a four-action policy-selection space, and a rewardfunction that penalises accumulated waiting and turnaroundtime while rewarding process completions and CPU utilisation.The agent is evaluated on four workload profiles—CPU-bound,I/O-bound, bursty, and mixed—generated from Poisson arrivalsand exponential or uniform burst distributions. Experimentalresults show that the agent learns a meaningful state-dependentpolicy, exhibiting a clear preference for Round Robin under highvariance workloads. Coarsening the state representation from3,125 to 243 discrete states raises state-space coverage from 2.34%to 12.35% for a fixed training budget, and the resulting policyremains competitive with the strongest static baseline on burstyand CPU-bound workloads, with a gap of 5.6% and 13.8% inaverage waiting time respectively. The framework is modularand reproducible, with a full-stack implementation comprising aFastAPI backend, PostgreSQL persistence, and a React dashboard.