A collaborative optimization framework based on multi-agent reinforcement learning is proposed for orderly charging at electric vehicle charging stations and coordinated interaction with the power grid, providing a technical reference for intelligent charging coordination under grid interaction and electromagnetic compatibility constraints.
Abstract
Uncoordinated charging of large-scale electric vehicles exacerbates peak-valley differences and voltage exceedance risks in the power grid, while existing scheduling methods still have limitations in distributed decision-making, dynamic pricing, and multiobjective balancing. These problems become more significant in charging station clusters where power-electronic converters, communication links, and complex electromagnetic operating environments jointly affect grid interaction stability. In this paper, a collaborative optimization framework based on multi-agent reinforcement learning is proposed for orderly charging at electric vehicle charging stations and coordinated interaction with the power grid. First, each charging station is modeled as an autonomous agent, and distributed environment modeling is realized based on local observation information and Markov decision processes. Second, a proximal policy optimization algorithm is used to generate a dynamic service fee multiplier in a continuous action space, which is combined with a demand elasticity module to form an adaptive pricing mechanism. Finally, a composite reward system integrating grid stability, operational revenue, and user satisfaction is developed, and multi-agent convergence training is achieved through parameter sharing and generalized advantage estimation. The results confirm the overall benefits of joint optimization in load shaping, economic performance, and robustness, providing a technical reference for intelligent charging coordination under grid interaction and electromagnetic compatibility constraints.
This paper presents a bilevel Reinforcement Learning (RL) framework for optimizing Electric Vehicle (EV) charging through price-mediated coordination between grid operators and charging stations. Unlike prior work relying on direct control or manual subgoal engineering, the proposed approach uses dynamic pricing as an implicit coordination signal to address a complex multi-objective optimization problem involving grid stability, user satisfaction, and economic efficiency. To manage this complexity, the problem is decomposed into two levels comprised of an upper-level Distribution System Operator (DSO) that determines dynamic pricing strategies, and multiple lower-level Load Aggregators (LAs) responsible for EV charging decisions at individual stations in response to these prices. This bilevel structure captures the leader–follower interaction between DSOs and LAs, with each level operating at different temporal scales. Deep Deterministic Policy Gradient (DDPG) agents are deployed at both levels, enabling adaptive decision-making under operational constraints. Extensive simulations compare the framework against multiple Rule-Based Control (RBC) baselines. Results demonstrate that the DDPG-based DSO achieves a 42.4% higher mean reward and 19.1% higher profit compared to the best-performing RBC baseline, while preserving grid stability and user satisfaction. These results validate the effectiveness of bilevel RL for complex energy optimization problems, highlighting its potential as a scalable control paradigm for smart management systems.
D. Vamvakas, Christos D. Korkas, E. Kosmatopoulos· Energies· 0 citations
The rapid growth of electric vehicles (EVs) introduces significant operational challenges to electric power grids, particularly in semi-connected and non-interconnected systems where monitoring and demand control capabilities are limited. Uncoordinated electric vehicle charging may increase peak demand and threaten grid stability. This paper proposes a negotiation-based multi-agent framework for managing EV charging while improving grid reliability and user satisfaction. The framework is implemented using the JADE multi-agent platform, where autonomous agents represent electric vehicle owners and parking lot operators. Parking lot agents generate dynamic pricing plans based on power demand levels and time-of-use tariffs, while electric vehicle agents submit charging requests according to arrival time, battery state, and user preferences. A call-for-proposal negotiation mechanism is used to generate mutually acceptable charging schedules between agents. The effectiveness of the proposed framework is evaluated through simulation studies based on a case study of the Palestinian power grid, characterized by semi-connected and non-interconnected network structures.
Ahmad N. Jallad, I. El-Sayed, Tala N. A. Hattab· 2026 6th International Confe...· 0 citations
Balancing markets require flexible resources that can promptly follow dispatch signals. Aggregated fleets of electric vehicles (EVs) operated as electric-vehicle virtual power plants (EV VPPs) are promising candidates. Aggregators must control the total power of EV chargers to track dispatch signals while satisfying individual EV users' charging demands. Conventional centralized optimization methods can achieve high tracking performance. However, they rely on global information and require solving large-scale optimization problems, which impose high computational and communication burdens and limit scalability. To address this issue, this paper proposes a two-level hierarchical control scheme based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. At the upper level, each charging station is modeled as an agent, and at the lower level its policy allocates charging and discharging power to the individual EV chargers. At runtime, each station-level agent uses only local observations and the broadcast dispatch signal. We present a case study on participation in Japan's balancing market Secondary 2 (S2) product. The study evaluates the controller on an EV VPP consisting of five stations with a total of 50 Level 2 chargers. The proposed method achieves a dispatch tracking rate (fraction of dispatch intervals with aggregate power inside the market-defined tracking error band) of 97 percent within the allowable tracking error band around the dispatch signal. It also achieves an 80 percent Target SoC satisfaction rate, where the Target SoC is the user-specified departure-time state of charge (SoC). Overall, this method reduces online computation time and communication latency while maintaining high tracking performance and userdemand satisfaction. These results suggest that MADDPG-based hierarchical control provides a practical control scheme for large EV fleets when latency constraints hinder centralized control.
Traditional single-objective strategies for coordinating electric vehicle (EV) charging and discharging are often unable to balance grid stability, user costs, and battery degradation simultaneously, limiting their applicability in intelligent energy management systems associated with modern electromagnetic power infrastructures. This challenge is particularly significant for industrial microgrids, such as those serving manufacturing facilities, where EV fleet integration must preserve the reliable power quality required by sensitive electrical equipment and electromagnetic energy systems. To address these issues, this study proposes an improved Multi-Objective Differential Evolution (MODE) algorithm that explicitly optimizes multiple conflicting objectives in parallel. The conventional differential evolution framework is enhanced through non-dominated sorting and crowding distance mechanisms to improve solution diversity and convergence. A coordinated EV scheduling model incorporating four optimization objectives, including user satisfaction, together with constraints on bus voltage, charging/discharging power, feeder thermal limits, and battery state of charge, is established. A two-dimensional matrix encoding strategy and an external Pareto archive are adopted to enhance optimization stability. Experimental results demonstrate that the proposed MODE approach achieves a grid load standard deviation of 6.95 and a peak-to-valley ratio of 1.63 while maintaining an average battery depth of discharge of 15% and a user satisfaction level of 0.92 for commuting scenarios. These findings verify the effectiveness of MODE for multi-objective coordinated scheduling and provide a practical optimization framework for sustainable EV energy management and industrial microgrids requiring stable electromagnetic power delivery.
The large-scale deployment of electric vehicles poses significant challenges for distribution grids, particularly in charging parks with limited network capacity. This paper proposes a fair and grid-aware charging management system based on a receding-horizon game formulation. The coordinated charging problem is modeled as a variational generalized Nash equilibrium (GNE), enabling the allocation of charging power among self-interested vehicles under shared grid constraints. Individual objectives capture price signals, charging smoothness, and terminal energy targets, while electricity network limits are explicitly enforced. In a practice-oriented weekly scenario with limited charging infrastructure, the method enforces the transformer constraint at all times and shows that a 2 h charging-time policy eliminates severe shortfalls above 16 kWh, at the cost of higher shortfalls for some vehicles. These results indicate that the proposed receding-horizon v-GNE formulation provides a transparent mechanism for grid-compliant and fairness-oriented charging coordination.
Benjamin Herrn, Felix Krabbes, Bernd Juris et al.· International Conference on...· 0 citations
This paper proposes a coordinated energy management framework for plug-in electric vehicle (EV) charging and discharging that minimizes operational cost while preserving grid stability under uncertain user behavior. The uncertainty of charging demand is represented through stochastic initial state-of-charge (SOC) levels, which capture the variability of EV energy requirements upon arrival. Based on this uncertainty representation, an optimization model incorporating battery dynamics, time-of-use (TOU) pricing, vehicle-to-grid capability, mobility constraints, and peak demand limits is formulated. To solve the resulting nonlinear optimization problem, the Grey Wolf Optimization (GWO) algorithm is employed and benchmarked against Genetic Algorithm (GA), Non-dominated Sorting Genetic Algorithm-II (NSGA-II), and Particle Swarm Optimization (PSO) Algorithm. Simulation studies conducted on a fleet of EVs over a 24-hour scheduling horizon demonstrate that the proposed framework maintains SOC within the safe operating range while ensuring that all vehicles satisfy the departure SOC target.
D. C. Huynh, Loc D. Ho, M. Dunnigan· 2026 6th International Confe...· 0 citations