reinforcement learning

Dynamic Topological Dependency Graph Learning (DTTL)

This paper introduces Dynamic Topological Dependency Graph Learning (DTTL), a novel approach to program dependency analysis that leverages reinforcement learning to automatically construct and refine dependency graphs without manual intervention. Traditional static dependency analysis tools are limited in their ability to adapt to the dynamic nature of software execution. DTTL addresses this limitation by integrating dependency learning directly into the program's runtime, using feedback from code execution to dynamically adjust and update the topological representation of dependencies. The core mechanism utilizes a reinforcement learning algorithm to learn optimal dependency relationships based on execution outcomes – success or failure, and performance metrics. The resulting graph accurately reflects the state of the program's execution and continuously evolves alongside the program's development. This approach offers significant improvements in accuracy and adaptability compared to static analysis, paving the way for more intelligent and responsive software development tools.

基于多智能体协作的复杂系统模拟

This paper presents a novel approach to simulating complex systems by leveraging the power of multi-agent collaboration. The core idea is to construct a population of intelligent agents, each with varying levels of cognitive ability, and task them with solving complex problems collectively. Reinforcement learning is employed to train these agents, enabling them to learn optimal collaborative strategies. The system's emergent behavior is then utilized for both prediction and control purposes. Unlike traditional single-agent simulation methods, this approach capitalizes on the distributed intelligence and information exchange among agents, leading to potentially more accurate and efficient system modeling. We demonstrate the feasibility and effectiveness of this methodology through a theoretical framework, outlining the key components and operational principles. The primary contribution lies in the systematic integration of multi-agent reinforcement learning with complex system simulation, offering a promising avenue for tackling systems exhibiting emergent behaviors and intricate interactions.

Dynamic Resource Allocation in Serverless Architectures via Reinforcement Learning

Serverless computing architectures offer significant advantages in terms of scalability and cost-efficiency. However, traditional serverless resource allocation strategies often rely on static configurations or simple heuristics, which can lead to suboptimal performance and increased costs. This paper proposes a novel approach to dynamic resource allocation in serverless environments using reinforcement learning (RL). The core idea is to train an RL agent to intelligently allocate serverless function resources—specifically, memory (M) and CPU (C)—in response to fluctuating demand. The agent observes system metrics such as function invocation rates, execution times, and error rates, and uses this information to learn an optimal policy for resource allocation. The objective is to minimize the total cost while maintaining a desired level of performance. The system utilizes a Q-learning algorithm to learn the optimal Q-values, representing the expected cumulative reward for taking a specific action in a given state. The agent's policy is then derived from these learned Q-values. Simulation results demonstrate that the RL-based approach significantly outperforms traditional allocation methods in terms of both cost reduction and performance improvement. The research highlights the potential of RL for optimizing resource utilization in serverless systems and contributes to a more efficient and responsive cloud computing landscape.

Development of a Collective Intelligence Framework for Optimising Demand and Supply-Side Smart Energy Management to Support Net-Zero Energy Outcomes in the Built Environment

Achieving net-zero energy in residential precincts remains challenging due to diverse energy demand patterns, intermittent renewable energy generation, and the need to balance energy demand, supply, thermal comfort, and affordability. Existing studies have focused on optimising demand and supply to reduce grid energy consumption and cost, but have given limited attention to the emergence of collective intelligence through real-time interactions among humans, smart appliances, and battery energy storage systems across precincts to support the achievement of net-zero energy outcomes. Furthermore, collective knowledge sharing among these elements remains underexplored. This research develops a collective intelligence framework for optimising demand- and supply-side energy management to support net-zero energy outcomes at the precinct level. The framework consists of: (1) the modelling of a society of intelligent agents representing humans, smart appliances, and battery energy storage systems distributed across the precinct; and (2) a Collective Transfer Reinforcement Learning (CTRL) methodology integrated with the society of intelligent agents to enhance collective intelligence. The CTRL methodology includes three modules: a Discovery Module for discovering successful energy-use experiences that map onto high-performance outcomes; an Advisory Module for transferring successful experiences among agents; and a Diversity Module for preventing over-reliance on transferred experiences and avoiding suboptimal solutions. The developed collective intelligence framework was applied in two case study precincts. The results show a near net-zero energy outcomes while maintaining thermal comfort, effectively utilising solar energy, and reducing peak electricity demand. Compared with optimisation methods without collective intelligence, it reduced total net energy consumption by 28% and grid electricity costs by 8.6% at the precinct level. The study develops a multidisciplinary approach that integrates the built environment and energy systems domains, along with advanced artificial intelligence techniques, to tackle grand challenges in achieving net-zero carbon goals. Unlike conventional approaches, the novel collective intelligence framework developed in this thesis enables collective intelligence to emerge through interactive learning and experience sharing among intelligent agents, thereby supporting net-zero energy outcomes at the precinct level.

Dynamic Topological Dependency Reasoning Network (DTTRN)

This paper introduces the Dynamic Topological Dependency Reasoning Network (DTTRN), a novel neural architecture designed for real-time learning and adaptation of complex system topology dependencies. Traditional dependency analysis methods rely on static, pre-defined topologies, limiting their effectiveness in dynamic and evolving systems. DTTRN addresses this limitation by employing a multi-layered, variable topology graph neural network where nodes represent system components and edges represent dependencies. Reinforcement learning is utilized to dynamically adjust edge weights and topology structure based on system behavior, optimizing overall performance and robustness. Furthermore, knowledge graph embedding techniques are integrated to enrich node representations with domain knowledge, enhancing reasoning accuracy. The core claim of this work is to build an architecture capable of real-time learning and adaptation of complex system topology dependency relationships, surpassing the constraints of static dependency graphs. DTTRN achieves this through a synergistic combination of graph neural network adaptability, reinforcement learning optimization, and knowledge graph embedding representation, resulting in real-time, self-adaptive reasoning for complex systems, robust to behavioral changes and benefiting from contextual knowledge for improved precision and generalization.

##基于生物反馈的机器学习模型优化

This paper explores the novel approach of utilizing biofeedback signals, specifically electroencephalogram (EEG) and electrocardiogram (ECG) data, to optimize the performance of machine learning models. The core idea is to leverage the dynamic and often subtle information contained within these physiological signals as training data for machine learning algorithms. We propose a framework based on reinforcement learning, where the biofeedback signal directly influences the adjustment of model parameters. This allows for a feedback loop where the model adapts to the user's internal state, potentially leading to enhanced accuracy and personalized model performance. The presented methodology offers a fundamentally new paradigm for machine learning optimization, moving beyond traditional supervised learning approaches and opening possibilities for adaptive and responsive systems. The research highlights the potential for improved model training and a deeper understanding of the relationship between human physiology and machine learning outcomes.

Adaptive Recursive Neural Network Architecture

This paper proposes a novel adaptive recursive neural network (ARNN) architecture designed to overcome the limitations of traditional, static recurrent neural networks. The core idea is to introduce dynamic adaptation of the network's recursive structure, leveraging evolutionary algorithms or reinforcement learning, to optimize learning efficiency and generalization performance. The system dynamically adjusts connection weights and hidden unit configurations within the recurrent layers based on network performance feedback. This contrasts sharply with conventional neural network design, which typically employs fixed architectures. Mathematical formulations and algorithmic descriptions are presented to detail the proposed architecture and the adaptive learning process. The presented approach aims to significantly improve the ability of RNNs to handle complex sequential data and achieve superior results compared to static RNNs. The primary contribution lies in the development of a robust and flexible framework for adaptive recurrent learning.

Traffic road control and lane allocation framework for intelligent transportation system using federated deep reinforcement learning

A Cognitive Architecture-Based General Artificial Intelligence Model

This paper proposes a novel approach to developing General Artificial Intelligence (GAI) by leveraging cognitive architectures. The core idea is to model human cognition—encompassing perception, memory, reasoning, learning, and decision-making—within a computational framework. This architecture is built upon a modular design, integrating deep learning and reinforcement learning techniques to enable the system to acquire and utilize diverse cognitive abilities. We argue that this approach, rooted in cognitive architecture, offers a more biologically plausible and potentially more effective path toward achieving true GAI compared to traditional, monolithic AI systems. The presented model aims to bridge the gap between artificial intelligence and human intelligence, ultimately leading to a system capable of adapting and solving complex problems across various domains. This document details the architecture, key mechanisms, and anticipated advancements within this framework. ---

Automated Theorem Proving with Deep Reinforcement Learning

Automated theorem proving (ATP) has achieved significant progress over the years, yet it frequently struggles with complex mathematical reasoning and strategic planning. This work introduces a novel approach to ATP leveraging deep reinforcement learning (DRL). The core idea is to train an intelligent agent capable of interacting with a theorem prover, learning optimal strategies for selecting theorems, axioms, and proof tactics. We hypothesize that a DRL agent can surpass human capabilities by learning a more efficient and robust reasoning process. This paper details the architecture of the agent, the reward function design, and the training methodology. Preliminary results demonstrate the potential of this approach to automate theorem proving effectively, suggesting a new paradigm for tackling challenging mathematical problems. The primary goal is to create an agent that can dynamically adapt to the proof process, improving upon the limitations of traditional, rule-based ATP systems.

Deep reinforcement learning for melt pool solidification cooling rate control in directed energy deposition

##基于强化学习的分布式系统资源调度

This paper investigates the application of Reinforcement Learning (RL) for dynamic resource scheduling in distributed systems. Traditional resource scheduling methods often rely on static rules or heuristics, which may not adapt effectively to the fluctuating demands and workloads of a distributed environment. This research proposes a novel approach where an RL agent learns to optimize resource allocation based on real-time system state. The agent interacts with the distributed system, observing resource utilization metrics and receiving rewards for efficient scheduling decisions. The core of this work lies in formulating the resource scheduling problem as an RL problem, employing algorithms like Q-learning or Deep Q-Networks (DQN) to train the agent. The ultimate goal is to achieve higher system throughput and improved resource utilization compared to traditional scheduling approaches. This work presents a theoretical framework and explores the potential of RL to dynamically adapt to the complexities of distributed system resource management.