FinRL-DeepSeek Revisited: A Reproduction and Multi-Seed Robustness Analysis of LLM-Infused Risk-Sensitive Trading Agents

Abstract

We reproduce and extend FinRL-DeepSeek (Benhenda, 2025), a trading framework combining Proximal Policy Optimization (PPO) (Schulman et al., 2017), Conditional Value-at-Risk (CVaR) constraints (Rockafellar and Uryasev, 2000), and large language model (LLM) signals extracted from financial news. Using the Nasdaq-100 universe (2013-2023) and the agent-ready datasets released by the original authors, we implement four agents from scratch in PyTorch - PPO, CPPO, PPO-DeepSeek, and CPPO-DeepSeek - and backtest them on the 2019-2023 period against an equal-weight buy-and-hold baseline. On a single seed, our results replicate the qualitative pattern reported by Benhenda (2025): LLM infusion improves both return and Sharpe ratio for both the unconstrained and CVaR-constrained agents. However, extending the evaluation to two independent seeds - a small but motivating step in light of well-documented seed sensitivity in deep RL (Henderson et al., 2018) - suggests that this single-seed pattern may not be stable, particularly for PPO-DeepSeek, whose Sharpe ratio ranking relative to plain PPO reverses between the two seeds we test. In contrast, CPPO-DeepSeek shows the smallest spread in total return across our two seeds of any agent. We present these observations as preliminary evidence, not a settled finding, and argue that they motivate wider-scale multi-seed evaluation in financial reinforcement learning research, an area where single-seed backtests remain the norm. We release our full codebase, including the trading environment, PPO/CPPO implementations, and evaluation pipeline, to support reproducibility.