The systematic development of single-agent to multi-agent ensemble systems shows great improvements in algorithmic and architecture of DRL-based portfolio management, and the research in the future focuses on the importance of explainable AI integration, meta-learning market regime adaptation, and consistent evaluation systems in reproducible research.
Abstract
Automated equity portfolio management is an important challenge in computational finance where the conventional optimization approaches fail to cope with non-stationary market forces and multifaceted decision-making processes. Deep reinforcement learning (DRL) algorithms have become a natural paradigm of sequential portfolio allocation decisions, providing their adaptive learning ability potentially more successful than traditional methods. This review will look at how automated portfolio manAdvanced Actoragement systems based on DRL have developed over time, beginning with single-agent applications, to more complex multi-agent ensemble systems. Our analysis of algorithmic advances, system architecture styles and technical implementation plans that have influenced this domain between 2018 and 2025. We have done a full search based on PRISMA practice by utilizing various scholarly databases, such as IEEE Xplore, ACM Digital Library, arXiv and Google Scholar. We used a search strategy based on deep reinforcement learning algorithms used in equity portfolio management with particular attention to technical applications and systems design. We obtained descriptive algorithm specifications, network structures, training procedures and performance indicators in 156 peer-reviewed articles. The discussion shows that there were four different evolutionary phases, with Phase I (2018–2020) consisting of simple single-agent models such as Deep Q-Networks (DQN) and REINFORCE with a low Sharpe ratio (0.7–1.0). In Phase II (2020–2022), improved phase II policies overcame phase I (2015–2017) results due to the implementation of novel advanced phase II policy gradient methods such as Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC) with Sharpe ratios of 1.2–1.4. The phase III (2022–2024) was the stage of development of multi-agent systems with cooperative, competitive, and hierarchical structures with a Sharpe ratio of 1.5–1.8. Phase IV (2024–2025) is the next horizon where the ensemble systems will incorporate a number of DRL algorithms and exhibit a Sharpe ratio of 1.8–2.4 and accrue a maximum drawdown of about 8%. The systematic development of single-agent to multi-agent ensemble systems shows great improvements in algorithmic and architecture of DRL-based portfolio management. The identification of the essential system design patterns, algorithmic developmental courses and technical implementation issues are some of the contributions. The research in the future focuses on the importance of explainable AI integration, meta-learning market regime adaptation, and consistent evaluation systems in reproducible research.
Ablation studies comparing the AHRL-PM model to a synchronous model and single-layer reinforcement learning (RL) approaches validated its superior performance in terms of profitability and risk-adjusted returns, while also highlighting significant reductions in training time and appropriate portfolio weight adjustments to respond to market dynamics and uncertainties, underscoring the model's efficiency and practical applicability.
Shuyu Liu, Tianxiang Cui, Yiran Li et al.· IEEE Transactions on Neural...· 0 citations
A closed-loop multi-agent decision framework that introduces prompt-level learning as a scalable alternative to full model retraining and highlights the potential of prompt-level adaptation for building robust and autonomous financial decision systems.
Kandarp Mukeshkumar Sharda, Aliyu Sani Sambo· NLP & Big Data· 0 citations
This paper introduces a dynamic portfolio optimization framework for large institutional investors using Scientific Physics-Informed Reinforcement Learning (SciPhyRL). Formulated in continuous time over an extended state space that includes explicit cumulative costs, the approach leverages offline historical data to learn optimal, distribution-aware strategies. A core innovation reduces the optimization challenge to solving an HJB equation by projecting it onto observed trajectories as a pathwise Hamilton-Jacobi equation. This is solved directly from data using PINN in a single offline sweep, eliminating the need for traditional value or policy iteration. To make the method effective at practical short horizons, the control variable is recast from a continuous trading rate to a discrete target holding. This ensures signal-implied positions are reached immediately, while execution costs are evaluated against a microstructure-grounded quadratic price impact model. Evaluated on a $14$-asset ETF universe using an engineered oracle signal, the learned Gibbs policy yields substantial out-of-sample Sharpe ratio improvements over static and myopic baselines. The results demonstrate that the proposed framework successfully translates known signal quality into a robust, multi-period, and cost-aware allocation mechanism with strictly controlled volatility and turnover.
The integration of Large Language Models (LLMs) with Reinforcement Learning (RL) for financial decision-making has grown rapidly in recent years, yet the literature remains fragmented and lacks systematic comparison across methods. In this survey we analyze 34 core studies (2023–2026), selected through a multi-stage process involving 84 initial candidates and 46 full-text reviews, and propose a three-paradigm taxonomy (feature-based, auxiliary-based, and policy-based) based on the functional role of LLMs within the RL pipeline. Analysis of these integration paradigms reveals an emergent architectural trade-off: while tighter policy-based coupling theoretically offers deeper contextual reasoning, it frequently introduces significant computational overhead and training instability. Conversely, simpler feature-based integration provides superior scalability and stability, though often at the expense of representational depth. Given the current benchmark fragmentation, the reported performance gains across these studies remain difficult to validate universally across different asset classes. Critical gaps identified include the insufficient handling of data leakage and look-ahead bias, standardized benchmarks, and limited alignment with regulatory frameworks such as MiFID II and the EU AI Act.
Ghusoon Hadi al-Aldaffaie, Alireza Taheri, Amirfarhad Farhadi et al.· Discover Artificial Intellig...· 0 citations
In a world of unparalleled market volatility and fragmented customer journeys, the old customer equity management models based on fixed segmentation and post-hoc analytics have not been sufficient to capture the dynamic development of customer-firm relationships. This paper presents an elaborate reinforcement learning (RL) model of dynamic customer equity optimization, which views marketing decisions as adaptive interventions that are sequential in non-stationary environment. To construct a practically implementable and theoretically based architecture of real-time marketing decision-making, we combine recent developments in the deep reinforcement learning, causal inference, and customer lifetime value (CLV) modeling. The framework combines: (1) multi-response state models that maintain Markov properties whilst learn online customer value signals; (2) conservative Q-learning to ensure reliable policy learning on offline data; (3) factor sensitive reward designs that include time varying customer engagement dynamics; and (4) multi-objective optimization that balances acquisition, retention and profitability goals. Empirical results on a variety of industry applications show that RL-based methods obtain significant improvements over constant baselines, and reported improvements in targeting efficiency of 27% (Qini coefficient), ROI gains of 18-58 and CLV impact gains of 45-85 (Wang and Chen, 2025). We cover theoretical background, issues in implementation and research directions in the future by arguing that dynamic customer equity optimization is a paradigm shift; instead of reactive, campaign-based marketing, dynamic customer equity optimization is proactive, relationship-oriented value co-creation. The paper ends by highlighting research gaps that are crucial to fill and outlining an agenda to further develop the combination of reinforcement learning and customer equity theory.
P. Khan, Muhammad Junaid, M. Ajmal et al.· International Journal of Bus...· 0 citations
Retail investors lack access to the kind of personalized, tax-aware portfolio management that institutional clients take for granted -- existing robo-advisors use static, rule-based allocation, and institutional-grade systems require account minimums and technology stacks unavailable to individual investors. We present a fully built, integration-tested application that closes this gap: a FastAPI backend and web dashboard that let a user describe an investment goal in plain language (e.g."I want steady growth but need to sell some shares next month for a down payment"), routes that goal to one of six investment mandates, and produces a live, broker-integrated portfolio recommendation from athree-phase reinforcement learning system -- a self-supervised cross-asset encoder, a Mixture-of-Experts (MoE) allocation policy with a learned intent router, and a lightweight LoRA adapter that personalizes recommendations from an individual's revealed brokerage behavior without retraining the shared model. The system is functionally complete and integration-tested end-to-end against a live brokerage API (Alpaca, paper-trading mode), including multi-user authentication, a trust first preview-before-apply confirmation flow, daily email digests, and an auditable action-integrity chain, but has not yet been opened to real end-users; we report this honestly as an emerging, pre-deployment application with a concrete path to full deployment, alongside 14-day walk-forward backtests (bootstrapped confidence intervals included) as preliminary, pre-deployment validation rather than production performance. We also report several practical engineering lessons -- silently-inactive integration paths, hanging third-party API calls, and the value of end-to-end empirical verification over trusting checkpoint metadata -- that we believe generalize to other applied RL systems built on external, live data sources.