Quantum computing holds immense promise for revolutionizing various fields, but the design and optimization of quantum circuits remain a significant challenge. Current methods often rely on manual tweaking and are limited by the time and expertise required. This paper introduces a novel system for dynamically reconfiguring quantum circuit parameters during runtime, driven by reinforcement learning. Our approach addresses the bottleneck in quantum circuit optimization by automating the process and providing real-time adjustments based on performance metrics. The system is designed to intelligently explore parameter space, maximizing performance while ensuring stability and minimizing resource consumption. This work presents a framework for automated circuit optimization, paving the way for more efficient and accessible quantum computing.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
Quantum simulation, the process of mimicking the behavior of quantum systems, holds immense promise for advancing scientific discovery in fields ranging from materials science to drug discovery. However, traditional quantum simulation methods often face challenges related to computational cost and fidelity, particularly when simulating complex systems with intricate dynamics. This paper introduces an adaptive quantum state collapse mechanism designed to address these limitations. We propose a reinforcement learning-based approach to dynamically adjust collapse criteria for quantum states, optimizing for both simulation accuracy and computational efficiency. The core mechanism utilizes a feedback loop that continuously assesses the system's internal state and adjusts collapse thresholds to achieve a balance between fidelity and resource utilization. We demonstrate the efficacy of this approach through simulations of a simplified, yet representative, system exhibiting dynamic quantum correlations. The resulting algorithm offers a potential pathway to significantly enhance the performance of quantum simulations.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper proposes a novel approach to system management leveraging dynamic topological information for intelligent inference. The core idea is to construct and maintain a semantic network that reflects the evolving relationships between system components, considering factors like network latency, device load, and communication costs. This network is continuously updated using a reinforcement learning-based probabilistic model, integrated with causal inference and knowledge graphs. The model learns and adapts to changes in the system topology, enabling real-time state prediction, anomaly detection, and optimized resource allocation. The system's performance is evaluated through simulation, demonstrating the effectiveness of the proposed methodology. The key contribution lies in treating topology as a dynamic element within the semantic reasoning process, moving beyond static parameter assumptions. The system utilizes a core claim of dynamic topology information to build and update a semantic network. The core mechanism involves a reinforcement learning-based probabilistic model combined with causal inference and knowledge graphs.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper investigates the application of multi-agent reinforcement learning (MARL) for optimizing complex systems. Traditional system optimization approaches often rely on centralized control or distributed control strategies, which can struggle with the inherent complexity and dynamic nature of many real-world systems. This research proposes a novel framework utilizing a population of intelligent agents trained through MARL to autonomously optimize these systems. The core concept involves decomposing the complex system into multiple agents, each responsible for optimizing a specific sub-objective. These agents interact through a combination of cooperation and competition, ultimately leading to overall system optimization. We explore the theoretical foundations of this approach, outlining the key components and the learning dynamics involved. The paper demonstrates the potential of MARL to overcome the limitations of conventional methods, offering a more adaptive and robust solution for complex system optimization problems. The effectiveness of this approach is discussed through a theoretical analysis and a conceptual design, paving the way for future research and practical implementations. ---
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Abstract Physical AI systems must reason about real-world dynamics in order to perceive, predict, and act safely under partial observability and uncertainty. World models–learned predictive representations of environment dynamics and action consequences–have emerged as a unifying framework for integrating perception, prediction, planning, and control in embodied agents. This survey provides a comprehensive and technically grounded review of learning-based world models for Physical AI, with particular emphasis on closed-loop decision-making. We organize existing approaches along six compositional design dimensions: state abstraction, temporal dynamics, uncertainty source and treatment, structural prior, observation modality, and decision coupling. Beyond this design-oriented taxonomy, we analyze how world models interact with optimization–highlighting compounding error, planner exploitation, rollout horizon management, and uncertainty calibration as central design tensions. We further examine evaluation methodologies, benchmark ecosystems, and sim-to-real transfer challenges, and synthesize open problems in long-horizon consistency, physical constraint enforcement, data efficiency, and safety. By clarifying recurring trade-offs across robotics and model-based reinforcement learning, this survey outlines principled directions for building reliable and scalable Physical AI systems.
Sven Kirchner, Nils Purschke, Alois Knoll· Discover Artificial Intellig...· 0 citations
This paper presents a novel approach to swarm control that leverages collective intelligence to achieve adaptive behavior. Traditional swarm control methods often rely on pre-defined rules and lack the flexibility to respond to dynamic environmental changes or unexpected events. This work addresses this limitation by constructing a control system for swarms where individual agents learn and adapt their behavior through interactions with their peers, informed by Bayesian inference and reinforcement learning. The core idea is to create a decentralized system where collective knowledge emerges, enabling the swarm to optimize its performance in complex and unpredictable scenarios. The system is designed to handle uncertainties and adapt to evolving task requirements. Mathematical formulations are provided to illustrate the key components of the control architecture, including agent interaction models, Bayesian inference processes, and reinforcement learning algorithms. This approach represents a significant step towards truly adaptive and robust swarm control, with potential applications in robotics, autonomous systems, and other areas where adaptability is paramount.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
ABSTRACT This paper proposes an adaptive optimal containment control method for nonlinear strict‐feedback multiagent systems with state constraints. First, a neural network‐based reinforcement learning algorithm is developed within an optimized backstepping framework. Unlike the traditional actor‐critic network structure, the paper introduces an observer‐actor‐critic architecture, where observers are used to estimate unmeasurable states, improving the reliability and accuracy of the system. Then, logarithmic barrier Lyapunov functions are combined with an optimal cost function to handle the state constraints. Using the Lyapunov stability theory, it is rigorously proven that all closed‐loop signals are uniformly, ultimately bounded, and that all system states remain within the constraint set. Finally, the proposed scheme is demonstrated to be valid using numerical and practical simulation examples.
Li Tang, Ben Niu, Ning Zhao et al.· International Journal of Rob...· 0 citations
Deep learning models often exhibit sensitivity to training data, leading to suboptimal performance. Traditional regularization techniques, while effective, can be inflexible and require extensive hyperparameter tuning. This paper introduces an adaptive topology for deep learning model regularization that dynamically adjusts the connectivity patterns within a neural network based on the characteristics of the training data. We propose a reinforcement learning-based algorithm to automatically optimize the topology, resulting in improved generalization and robustness. The core mechanism leverages the concept of a dynamically evolving network structure to mitigate the effects of data heterogeneity. The proposed approach offers a novel and potentially transformative solution for enhancing deep learning model performance.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper introduces a novel self-adaptive reinforcement learning algorithm based on topological networks. Traditional reinforcement learning approaches often rely on predefined reward functions and static policies, limiting their ability to adapt to dynamic environments. Our algorithm leverages the inherent topological structure of the environment to enable more intelligent and adaptive decision-making. We propose a method for constructing a topological representation of the environment, employing a hierarchical representation to capture complex relationships and dependencies. The core mechanism involves a dynamically adjusted policy based on this topological structure, allowing the agent to efficiently explore and react to changes in the environment. We demonstrate the effectiveness of this algorithm through several illustrative scenarios, showcasing its ability to surpass traditional reinforcement learning methods in terms of sample efficiency and robustness. The proposed algorithm provides a promising avenue for developing more sophisticated and adaptable reinforcement learning agents capable of navigating complex and evolving environments.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
This paper presents a novel approach to neural network optimization that dynamically adapts the network topology based on learned dependencies. The core idea is to leverage reinforcement learning, where two agents collaborate: one adjusts connection weights and the other modifies the network's structure (adding, removing, or restructuring connections). A 'dependency graph' guides the agents' decisions, reflecting the learned information dependencies between neurons. This dynamic adaptation addresses the limitations of traditional methods that assume a fixed network topology, particularly when dealing with complex dependencies and non-Euclidean data. The system aims to achieve more efficient training and improved generalization performance by allowing the network to evolve its structure to better represent the underlying data. The optimization process is driven by minimizing a loss function, and the dependency graph is continuously updated based on the error signal. The key contribution lies in the integration of topology adaptation with reinforcement learning, providing a framework for creating inherently adaptive and robust neural networks. The proposed methodology demonstrates potential for significant improvements in network performance across various domains.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
Purpose Edge-AI intrusion detection in cyber-physical-social systems (CPSS) must operate under temporal drift while respecting strict low-false-positive-rate (low-FPR) budgets. This study aims to evaluate whether a deep reinforcement learning intrusion detector, RO-DDDQN, provides a reliable low-FPR operating point under a strict temporal protocol. Design/methodology/approach The authors study a cost-sensitive Double Dueling Deep Q-Network with a GRU-based temporal encoder (RO-DDDQN) on CSE-CIC-IDS2018 under a strict day-level split (train: Feb. 14–20; validation/calibration source: Feb. 21; test: Feb. 22). Three thresholding protocols are compared at target FPR = 1%: naive validation-tail thresholding, deployable benign-tail thresholding and a non-deployable test-oracle diagnostic. RO-DDDQN is further compared with an edge-suitable XGBoost baseline under the same temporal protocol and robustness is assessed across five random seeds. Findings For the seed-42 run, validation-based threshold selection is highly misleading under drift: a threshold tuned to 1% FPR on validation inflates to 8.62% FPR on the future test day. Benign-tail calibration does not repair RO-DDDQN deployability: at a nominal 1% target, the achieved FPR rises to 9.10% with 53.39% recall. Even under a test-oracle threshold, this run reaches only 1.99% recall at 1% FPR. Multi-seed analysis shows that this outcome is not representative: across five seeds, RO-DDDQN achieves zero attack recall at oracle 1% FPR in four of five runs, with median oracle recall of 0.00%. By contrast, XGBoost generalizes more effectively (ROC-AUC 0.9736) and achieves 35.06% recall at 1.16% FPR under deployable calibration. Research limitations/implications This study is limited to one benchmark (CSE-CIC-IDS2018), one strict temporal split, one short-horizon windowing setup and one value-based DRL formulation (RO-DDDQN). Results should therefore not be interpreted as universal conclusions about all reinforcement-learning-based IDS methods or all CPSS telemetry settings. The experiments also do not include a full uncapped-window ablation and broader statistical characterization remains limited despite the added five-seed analysis. Future work should extend evaluation across rolling-origin days, additional datasets/traces and wider ablation/statistical studies. Practical implications For edge deployments operating under strict false-alarm budgets, validation metrics alone are insufficient. Practitioners should calibrate thresholds on benign-representative background traffic, explicitly test calibration transfer to future periods, run oracle and tail-overlap diagnostics to distinguish threshold mismatch from ranking collapse and assess robustness across random seeds. These checks reduce the risk of false-alarm inflation and help ensure that IDS models remain operational under temporal drift. Social implications In CPSS settings such as smart cities and critical infrastructure, unreliable intrusion alerts can waste scarce edge resources, desensitize operators and trigger unnecessary automated responses that affect services and users. More realistic evaluation of low-FPR behavior under drift supports safer deployment by reducing avoidable disruptions and improving trust in security monitoring systems that may influence cyber-physical decisions. Originality/value The study contributes a temporally faithful low-FPR evaluation protocol that separates threshold-transfer failure from ranking degradation under drift. The results show that, under the evaluated protocol, RO-DDDQN is highly seed-sensitive and does not provide a reliable low-FPR operating point, whereas calibration-transfer analysis and recall-at-fixed-FPR evaluation provide a clearer view of deployability. The findings are limited to one benchmark, one strict temporal split, one windowing setup and one DRL formulation, motivating broader rolling-origin and cross-dataset evaluation.
Lakhdar Kachna, Noureddine Chaib· International Journal of Per...· 0 citations
This paper proposes a novel cognitive architecture for embodied artificial intelligence (AI) systems, designed to address the limitations of current approaches. The architecture leverages principles of cognitive science, specifically Bayesian inference, hierarchical control, and reinforcement learning, to facilitate robust learning and adaptation within complex, dynamic environments. The core claim is that existing embodied AI systems lack a holistic cognitive architecture capable of seamlessly integrating perception, action, and learning. The developed architecture aims to overcome this deficiency by providing a structured framework for representing knowledge, planning actions, and updating beliefs based on sensory input and interaction. Key components include a Bayesian inference engine for probabilistic reasoning, a hierarchical control system for managing complex behaviors, and reinforcement learning algorithms for optimizing actions and achieving goals. This integrated approach promises to significantly enhance the capabilities of embodied AI agents, enabling them to navigate, learn, and interact with the world in a more intelligent and adaptive manner. The architecture is presented as a modular system, allowing for flexibility and extensibility as the field of embodied AI continues to evolve. This document outlines the architectural design, the underlying principles, and the anticipated benefits of this approach.
Jincheng Zhang· Zenodo (CERN European Organi...· 0 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
