Abstract The rapid adoption of renewable energy sources has radically changed the dynamic qualities of the modern power system, making load frequency control (LFC) problem worse due to unforeseeable generation, reduced system inertia, and the constant changes in disturbances. This paper presents a control framework based only on Proximal Policy Optimisation (PPO) for secondary frequency management in renewable-integrated power systems subjected to realistic multi scenario disruptions. In contrast to traditional gain-tuned or deterministic intelligent controllers, the suggested method immediately acquires an effective frequency regulation policy via policy gradient reinforcement learning, devoid of supplementary heuristic tuning. The controller is carefully evaluated under four pertinent operating conditions namely renewable intermittency, long cycle disturbances, oscillatory conditions of stress and a realistic contingency with renewable outages, night peak loads and gradual recovery. The performance is compared to GA-PI, TS-Fuzzy-PI, GA-Fuzzy-PI and advanced deep reinforcement learning-based controllers DDPG, TD3 and SAC on the basis of frequency domain and time-domain stability. The proposed PPO architecture shows a high improvement in the frequency control performance. In Case 1, the Integral Square Error (ISE) is reduced to 0.45196, as opposed to approximately 1060 with benchmark controllers. At the presence of high cyclic variations (Case 2), the ISE reduces to 0.031126, but the mean amplitude change of the frequency is smaller, 0.00043351 Hz, compared to approximately 0.3125 Hz in other methods. In the realistic contingency (Case 4), an Integral Squared Error (ISE) of 0.10953 and a constrained overshoot of 0.023203 Hz are attained. The findings validate that PPO provides exceptionally resilient, adaptable, and cost-effective frequency stabilisation for renewable-dominated power systems under actual dynamic stress circumstances.
Language models are typically trained to predict the next token in a sequence. Here, we explore an alternative predictive principle from reinforcement learning: Successor Representations (SRs), which model the expected discounted distribution of future states rather than the immediate next state. We transfer this framework to natural language and train neural networks to predict future word distributions across multiple temporal horizons, thereby learning representations of long-range transition structure. We train a deep residual neural network on WikiText-103 (103 million tokens; 20,000-word vocabulary) and optimize successor representations as probability distributions using KL divergence. Without explicit linguistic supervision, structured language representations emerge spontaneously. After training, the learned space develops a clear geometric organization with respect to part-of-speech (POS) categories: nouns, verbs, and adjectives become separable and recoverable through unsupervised clustering. This organization depends systematically on predictive horizon, with short horizons producing the strongest syntactic structure and longer horizons increasingly integrating broader contextual and semantic information. At finer resolutions, additional interpretable lexical substructure emerges, revealing coherent subclasses within major word categories. These findings suggest that syntactic categories need not be explicitly encoded but may arise as a consequence of predictive sequence learning. To our knowledge, this work provides the first systematic application of successor representations to natural language and establishes a conceptual bridge between reinforcement learning, linguistics, and cognitive neuroscience.
This work presents a reproducible empirical comparison of Direct Preference Optimization (DPO) and Reinforcement Learning from Human Feedback (RLHF) for Large Language Model alignment. Experiments are conducted on GPT-2 (124M parameters) using the Anthropic HH-RLHF dataset. The study evaluates alignment quality, reward accuracy, training efficiency, inference latency, and alignment tax under consumer hardware constraints. DPO achieves 71% reward accuracy and a reward margin of 0.640 without requiring a reward model. All experiments are reproducible on an NVIDIA RTX 3050 6GB GPU using open-source tooling. Source code and experimental artifacts are available at:https://github.com/AnthropicBots/dpo-vs-rlhf-alignmet-study
Mohit Yadav· Zenodo (CERN European Organi...· 0 citations
In-pipe turbines have emerged as a promising solution for harnessing surplus energy in water transmission networks for distributed power generation. While much of the previous research has focused on optimizing the turbine itself, practical implementation presents additional challenges. To address these challenges, this study proposes a hydraulic scheme that integrates an in-pipe turbine with accompanying control valves, intended to replace conventional pressure regulation valves. This system not only regulates pressure in water distribution networks but also functions as a power generation unit. The utilized turbine in this study is a modified version of previously introduced drag-based in-pipe turbines, demonstrating significantly higher efficiency. The study proposes a dynamic control method to optimize turbine performance under transient hydraulic conditions of pipelines, based on reinforcement learning. This hydraulic control algorithm successfully adapts to new scenarios, achieving desired power generation while maintaining the pressure constraints of the water distribution network at various flow conditions. When tested on a benchmark network, the trained model can recover up to 40% of the energy that would be otherwise dissipated by a pressure-reducing valve or left unused. The proposed methodology in this study enhances the feasibility and reliability of in-pipe turbines by integrating their prior advancements in the design and optimization with an RL-based framework for their optimal deployment in water transmission networks.
Arian Shahnoori, Nima Hasanzadeh, Amir F. Najafi· Energy Reports· 0 citations
This paper proposes a novel approach to neural-symbolic reasoning by introducing a Dynamic Topological Memory Network (DTMN). The core idea is to construct a memory network capable of dynamically adjusting its internal topology to mirror the complexity and relationships within the input data. This addresses a key limitation of existing neural-symbolic methods that often rely on static knowledge graphs or predefined rules, struggling with uncertainty and intricate relationships. The DTMN incorporates a graph-based neural architecture where nodes represent concepts or facts, and edges represent their relationships. A "topological learner" dynamically adjusts the network's structure—including adding, removing, or modifying nodes and edges—using techniques like reinforcement learning or evolutionary algorithms, guided by the input data and existing knowledge. A "symbolic reasoning engine" then leverages this evolving topology for logical inference, generating symbolic expressions as output. The significance lies in the adaptive nature of the network's topology, enabling a more flexible and effective reasoning process compared to traditional methods. We demonstrate the potential of this architecture for robust and efficient symbolic reasoning tasks.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper investigates the optimization of distributed resource scheduling using a multi-agent reinforcement learning (MARL) framework. Traditional resource scheduling methods often rely on static rules or centralized control, which can be inflexible and inefficient in dynamic environments. This research proposes a novel approach where resource scheduling is formulated as a multi-agent reinforcement learning problem. Each agent is responsible for allocating a portion of resources, and they learn collaboratively through reinforcement learning to achieve global optimality. The system adapts dynamically to changing conditions, offering a more efficient and robust solution compared to traditional methods. The key contributions of this work lie in the application of MARL to resource scheduling, demonstrating its potential for improved performance and adaptability. The model is defined as (St, At, Rt, Pt), where St is the state, At is the action, Rt is the reward, and Pt is the transition probability.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper investigates the application of Deep Reinforcement Learning (DRL) for intelligent traffic signal control. Traditional traffic signal control methods often rely on pre-defined rules or simple optimization algorithms, which may not effectively adapt to dynamic traffic conditions. DRL offers a promising approach by allowing an agent to learn optimal control policies through trial and error interactions with the traffic environment. This research proposes a DRL framework for dynamic traffic signal control, aiming to mitigate congestion and improve traffic flow. The framework utilizes a deep neural network to approximate the Q-function, enabling the agent to learn complex traffic patterns and adapt its control strategy accordingly. The effectiveness of the proposed approach is evaluated through simulations, demonstrating its potential to outperform conventional control methods. The core claim of this work is to leverage DRL algorithms to optimize traffic flow and reduce congestion. The central mechanism involves modeling traffic signal control as a DRL problem and utilizing a deep neural network to learn optimal control policies. This represents a novel approach to traffic management, aiming to enhance optimization efficiency.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper introduces a novel framework for reinforcement learning, termed "Adaptive Heterogeneous Networks" (AHN), designed to dynamically adjust learning strategies based on the environment's dynamic characteristics. Traditional reinforcement learning often relies on static policies, limiting adaptability. AHN leverages an evolving network of interconnected modules, allowing the agent to automatically adjust its learning approach to optimize performance across varying conditions. We present a comprehensive analysis of AHN's architecture, training procedure, and performance evaluation, demonstrating its ability to significantly enhance learning efficiency and generalization capabilities compared to existing methods. The core mechanism centers around the construction and utilization of an adaptive heterogeneity network, enabling the agent to respond to evolving environmental states in a flexible and efficient manner.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper presents a novel algorithm for optimizing the structure of self-organizing neural networks (SONNs) through dynamic adjustment of connection strengths. Traditional SONNs often rely on manually defined layer configurations, which can be suboptimal and require significant tuning. Our proposed algorithm, termed "Adaptive Resonance Network Optimization" (ARNO), employs a self-adaptive reinforcement learning approach to automatically adjust connection weights, leading to improved network performance across various tasks. We demonstrate the effectiveness of ARNO through extensive experiments on benchmark datasets, showcasing significant gains in accuracy and efficiency compared to baseline methods. The core mechanism involves iteratively adjusting connection strengths based on a learned reward function, enabling the network to converge to an optimal configuration.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
