reinforcement learning

Dynamic Probability Field Model

This paper presents a novel dynamic probability field model designed to simulate complex system behavior and predict future states by dynamically adjusting probability field parameters based on real-time data and environmental changes. Leveraging the integration of machine learning and reinforcement learning, this model offers a significant advancement over traditional probabilistic models, enabling automated parameter tuning and real-time system modeling. The core mechanism focuses on continuously adapting probability field configurations, leading to enhanced prediction accuracy and efficiency. This work addresses a critical need for systems capable of robust and adaptable behavior in dynamic environments.

Dynamic Neural Network Topology (DNTN)

This paper introduces the Dynamic Neural Network Topology (DNTN), a novel neural network architecture designed to overcome the limitations of static, connection-based networks. The core claim of this work is that by dynamically adjusting the physical connection strengths and topology of neurons in real-time, adaptive learning and memory capabilities can be achieved, surpassing the constraints of traditional neural networks. The proposed DNTN utilizes a Microelectromechanical Systems (MEMS) array as a neuron substrate, with each MEMS structure representing a neuron. A reinforcement learning controller dynamically adjusts connection parameters based on task objectives and environmental feedback, optimizing network structure and function. Furthermore, a metabolic module mimics biological neuron energy consumption, preventing excessive connections and network degradation. The DNTN represents a significant advancement in neural network design, offering enhanced learning efficiency, flexibility, and a closer simulation of biological neural systems. Key characteristics include dynamic topology reconfiguration, real-time adaptation, and a biologically inspired metabolic control mechanism.

Automated Theorem Proving with Neural Network Guidance

Automated theorem proving (ATP) aims to develop systems capable of mechanically proving mathematical theorems. Despite significant advancements, ATP systems often struggle with complex reasoning tasks, largely due to the inherent difficulty in representing and executing logical deduction rules. This work proposes a novel approach to ATP that integrates neural networks to provide guidance during the proof process. The core idea is to train a neural network to suggest promising proof steps and identify relevant theorems, essentially acting as an "intelligent assistant" for the ATP system. This guidance mechanism is expected to improve the efficiency and effectiveness of ATP, particularly in tackling challenging mathematical problems. The presented framework utilizes a reinforcement learning approach, where the neural network learns to optimize the proof strategy based on the current state of the proof and the available theorems. The system is evaluated conceptually, outlining the architecture and training process, and highlighting potential improvements. Further research will focus on developing and refining the network architecture, exploring different training strategies, and integrating the guidance mechanism with existing ATP systems.

Bringing Reinforcement Learning to Multi-Period Financial Planning: A Bridge Between Learning-Enabled and Stochastic Optimization

Abstract Financial planning is rarely a one-shot decision: today’s saving, spending, and investment choices shape tomorrow’s wealth, liabilities, and goal attainment. This survey traces the mathematical evolution of multi-period financial planning over the past several decades, focusing on selected key methods ranging from classical stochastic optimization to learning-enabled decision systems. We begin with Markowitz’s single-period mean–variance optimization and trace the field’s evolution toward two major multi-period optimization paradigms: scenario-based approaches, such as multi-stage stochastic programming (MSP), and state-space approaches, including model-based dynamic programming (DP) and model-free reinforcement learning (RL). While traditional MSP and DP provide the mathematical backbone for sequential financial decision-making, their practical application has long been constrained by the curse of dimensionality, model misspecification, and restrictive assumptions required for tractability. Recent advances in model-free RL create new opportunities for adaptive and scalable sequential decision-making without relying on rigid transition models or handcrafted scenario trees. Building on this foundation, we review recent RL applications in multi-period financial planning for both individuals and institutions and classify the literature into three roles: hybrid RL, scalable RL, and end-to-end RL. Finally, we discuss the evolving landscape of AI/ML-driven automated investing, highlighting its promise for algorithmic financial planning as well as key challenges regarding data, interpretability, trust, and regulation.

AXIS Project: Proactive Biological Homeostasis via the ERSA Algorithm

Abstract: Biological organisms are open systems operating under rigorous thermodynamic constraints, where homeostatic maintenance requires a constant expenditure of energy to suppress entropic decay. Conventional medical interventions are primarily reactive, acting only after homeostatic collapse exceeds critical thresholds. We introduce the AXIS Project, a cybernetic framework for proactive biological regulation. Central to this initiative is the Entropy-Regulated Stabilization Algorithm (ERSA), which defines a 3D state space composed of physiological age ($A$), information entropy ($S$), and metabolic energy ($E$). By integrating reinforcement learning with Gompertz-Makeham nonlinear dynamics, ERSA identifies system-specific "Golden Setpoints" and preemptively stabilizes the system before metabolic collapse occurs. We demonstrate that ERSA effectively identifies collapse thresholds via explicit state transition modeling, providing an OS-level logic for precision bio-engineering and edge-embedded medical control. Research Context: The AXIS Project addresses the fundamental problem of biological decay through the lens of information thermodynamics, building upon the principles of Information-Entanglement Stabilization (IESA). This framework evaluates the ERSA model not as a physical intervention, but as a high-level software abstraction architecture optimizing multi-objective constraints under finite computational resources. Keywords: AXIS Project; ERSA Algorithm; Information Thermodynamics; Proactive Homeostasis; Gompertz-Makeham Dynamics; Computational Biology; Bio-engineering; Edge-embedded Systems

Tribological Performance of S-Glass and Graphene Reinforced Al6061 Composites: Experimental and Machine Learning Perspectives

PyCFRL: A Python library for counterfactually fair offline reinforcement learning via sequential data preprocessing

Reinforcement learning (RL) aims to learn and evaluate a sequential decision rule, often referred to as a "policy", that maximizes the population-level benefit in an environment across possibly infinitely many time steps. However, the sequential decisions made by an RL algorithm, while optimized to maximize overall population benefits, may disadvantage certain individuals who are in minority or socioeconomically disadvantaged groups. To address this problem, we introduce PyCFRL, a Python library for ensuring counterfactual fairness in offline RL. PyCFRL implements a novel data preprocessing algorithm for learning counterfactually fair RL policies from offline datasets and provides tools to evaluate the values and counterfactual unfairness levels of RL policies. We describe the high-level functionalities of PyCFRL and demonstrate one of its major use cases through a data example. The library is publicly available on PyPI and Github (https://github.com/JianhanZhang/PyCFRL), and detailed tutorials can be found in the PyCFRL documentation (https://pycfrl-documentation.netlify.app).

Analytical coarse grained potential parameterization by reinforcement learning for anisotropic cellulose

Purkinje cells in Crus I and II encode the visual stimulus and the impending choice as monkeys learn a reinforcement based visuomotor association task

Abstract Visuomotor association involves linking an arbitrary visual cue to a well-learned movement. Transient inactivation of Crus I/II impairs primates’ ability to learn new associations and delays motor responses without affecting the kinematics of the movement. The simple spikes of Purkinje cells in the Crus regions signal cognitive errors as monkeys learn to associate specific fractal stimuli with movements of the left or right hand. Here we show that as learning progresses, the simple spike activity of individual neurons becomes more selective for stimulus-response associations, with selectivity developing closer to the appearance of visual stimuli. Initially, most neurons respond to both associations, irrespective of the identity of the stimulus and the associated movement, but as learning advances, more neurons distinguish between specific stimulus-hand associations. Using a linear decoder, it was found that in early learning stages, the visual stimulus can be decoded only when the choice can also be decoded. As learning improves, the visual stimulus is decoded earlier than the choice. A simple model can replicate the observed simple spike signals and the monkeys’ behavior in both the early and late learning stages.

Deep reinforcement learning-based optimization of reconfigurable intelligent surfaces (RIS) for 6G multi-user connectivity in NLOS environments

Abstract The transition towards the sixth generation (6G) of wireless networks requires ultra-high data rates and seamless connectivity, even in frequency bands like mmWave and THz, which are highly susceptible to physical blockages. RIS have emerged as a key technology in mitigating NLOS limitations through programmable steering of electromagnetic waves by passive reflection. However, real-time optimization of high-dimensional RIS phase shifts in dynamic multi-user environments remains an NP-hard challenge that is hardly solvable efficiently by traditional methods based on mathematical optimization. This paper presents a new DRL framework, utilizing a TD3 architecture, for the optimization of sum-rate performance of multi-user links. The model incorporates real-time environmental feedback and imperfect CSI for ensuring robust connectivity. Simulation results show that the DRL-RIS framework proposed here achieves remarkable gains in the sum-rate performance and outperforms conventional convex optimization baselines in computational latency and energy efficiency. Keywords: 6G Networks, Reconfigurable Intelligent Surfaces (RIS), Deep Reinforcement Learning (DRL), Non-Line-of-Sight (NLOS), Beamforming

Hierarchical thermal management integrating LSTM prediction and DDPG reinforcement learning for cascade waste heat recovery in a methanol reforming PEMFC ship

Experimental Push-Off Test Dataset for Shear Transfer Capacity in GFRP-Reinforced Concrete Interfaces

This dataset compiles experimental push-off test results for evaluating the shear transfer capacity of concrete interfaces reinforced with Glass Fiber-Reinforced Polymer (GFRP) reinforcement. The database includes geometric, material, and reinforcement-related parameters used to characterize the tested specimens, including interface shear area, maximum aggregate size, reinforcement ratio and configuration, GFRP bar diameter, tensile strength and elastic modulus, concrete compressive strength, and experimentally measured shear transfer capacity. The dataset was assembled from published experimental studies and was used for the development and evaluation of machine-learning and regression-based predictive models for GFRP-reinforced concrete interfaces. It accompanies the study “Data-Driven Prediction of Shear Transfer Capacity in GFRP-Reinforced Concrete Interfaces” and supports reproducibility, model development, comparative assessment, and future research on shear transfer behavior of GFRP-reinforced concrete interfaces.