The results support the feasibility of direct language-model token generation for financial numerical prediction and decision-making, while motivating broader tests across assets, regimes, and random seeds.
Abstract
Financial prediction typically relies on task-specific regression, ranking, or policy heads, separating the language model from the numerical object ultimately evaluated. We investigate whether a causal language model can instead represent forecasts and decisions directly through constrained token generation. FinATOM introduces a unified, head-free interface for three-step stock-return forecasting and dynamic five-ETF allocation. The forecasting model autoregressively emits volatility-standardized return tokens and is trained with ordinal and ranking supervision followed by a one-epoch token-level policy stage. The allocation model generates normalized long-only weights; supervised fine-tuning imitates a causal mean--variance anchor, and DAPO-augmented GRPO optimizes realized 21-day Sharpe subject to anchor consistency. In 2023--2025 ETF tests, the allocation policy improves pooled gross Sharpe from 1.428 to 1.529 and net Sharpe under a 5-bp transaction-cost model from 1.394 to 1.494. The multimodal allocation input attains the highest three-period mean Sharpe of 1.540, with its clearest advantage in 2025. On FinTexTS, the SFT and policy strategies achieve 73.52\%/2.68 and 73.72\%/2.69 cumulative-return/Sharpe, respectively. These results support the feasibility of direct language-model token generation for financial numerical prediction and decision-making, while motivating broader tests across assets, regimes, and random seeds.
Large-scale targeting and recommendation systems are typically built around predictive scores fed into heuristic or local allocation. When the business goal is incremental impact, as in marketing campaigns, incentives, and notifications, this paradigm systematically misallocates resources toward users who would have acted anyway. We present a decision-centric framework that instead optimizes causal effects under global constraints, aligning three components under a single objective: a causal neural network with a Transformer backbone for individual treatment-effect estimation, a Bayesian neural-bandit layer for uncertainty-aware exploration, and a dual-based large-scale linear-programming layer for constrained allocation. The framework also supports sequential context and multi-outcome, attribute-conditioned scoring through a Transformer encoder and outcome embeddings. We evaluate it with offline simulations on a public bandit dataset, targeted architectural ablations, and an online A/B test on LinkedIn Feed marketing traffic. We also distill production lessons on causal training-data construction and cost and delivery control, which were critical to successful deployment. The end-to-end treatment policy delivered a statistically significant $+7.20\%$ lift in the primary long-term-value metric, demonstrating the feasibility of production-scale causal optimization under business constraints.
Changshuai Wei, John Bencina, Phuc Nguyen et al.· 0 citations
We present a language-model forecasting system for merger arbitrage, a specialized high-stakes financial setting in which the task is to predict the outcome of announced M\&A deals. Unlike prior work on judgmental forecasting with LLMs, which has focused on broad mixed-topic benchmarks and short context such as news snippets, we study a setting that requires long-context reasoning over hundreds of pages of technical documents. Our system combines expert-guided context engineering with finetuning on hindsight-guided reasoning traces derived from historical deals. Given an announced deal, it outputs a probability distribution over three mutually exclusive outcomes: closing at announced terms, a higher bid, or deal termination. On an out-of-sample set of more than 400 large deals spanning 42 countries, our finetuned system achieves the best performance of any method we evaluate, reducing class-balanced Brier score to 0.151. This is 24\% below calibrated market-implied probabilities, 19\% below XGBoost, and 25-42\% below frontier language models. These results, together with ablation studies, show that LLM-based forecasting can succeed in specialized, long-context financial workflows, with hindsight-based supervision and expert-designed context playing a critical role.
Hinal Jajal, Michał Mucha, C. Sweat et al.· 0 citations
Accurate prediction of financial time series is still a difficult problem as financial markets display high volatility, non-linearity and stochasticity. Traditional forecasting methods necessitate extensive domain knowledge in designing technical indicators for subsequent analysis, often resulting in the loss of intricate time dependencies. The goal of the present study is to propose a framework allowing for learning representations automatically from raw financial data that are informative in downstream forecasting tasks. The proposed framework, contrasting predictive coding (CPC), is based on self-supervised representation learning. The learned embeddings are applied to Linear Regression, Random Forest and LSTM to predict the next-day log returns of three major foreign exchange currency pairs: EUR/USD, GBP/USD and USD/JPY. Evaluating the Performance of CPC-Generated Representations and Conventional Handcrafted Features on Forecasting Models trained on Historical Market Data. The LSTM with CPC context embeddings produces the best overall performance with a drop in mean squared error of 18%, directional prediction accuracy of roughly 59%, and better risk-adjusted trading performance with Sharpe ratios above 0.7. Additionally, the outcomes of transfer learning experiments reveal that a CPC encoder trained using one currency pair efficiently generalizes to other currency pairs. The results indicate that self-supervised representation learning can serve as an effective and scalable substitute for manual feature engineering in finance time-series forecasting.
Muskan Pawar· International Journal of Ada...· 0 citations
Small and medium-sized businesses need timely financial guidance, yet historical accounting logs record self-selected and often co-occurring business changes rather than randomized recommendations. We formulate this setting as observational policy ranking: from pre-decision financial information, a policy selects one of 34 ledger-derived business-change categories for a target financial KPI. Using 85,078 company-month observations from 7,505 firms, we introduce Covariate-Adjusted Residual Policy Learning (CAR-PL), an action-wise R-learner that operates directly on multi-hot logs and regularizes selection by observational support. We compare CAR-PL with an uplift T-Learner, a conservative contextual value model, a zero-shot LLM, and non-personalized references on company-disjoint held-out firms under a shared model-assisted scoring rule. CAR-PL has the highest Gross Profit point estimate (0.084), the T-Learner has the highest Revenue point estimate (0.085), and the contextual value model has the highest Quick Ratio point estimate (0.062). CAR-PL and the T-Learner are not statistically separated on either growth KPI in matched company-clustered comparisons, while CAR-PL selects 33-34 categories and produces less concentrated selections across the catalog. Outcome-model-only scoring retains the same KPI-level point-estimate leader or top pair, and category rankings remain similar when the all-zero treatment reference is replaced by the most common training co-action pattern. These findings support objective-specific ranking of SMB financial guidance from multi-action accounting logs.
Shrutendra Harsola, Vignesh T. Subrahmaniam, Vikas Raturi et al.· 0 citations
Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic inputs. We introduce Vector-Input Autoregressive Inference for Ordinal-Return Modeling (VAIOM), a decoder-only Transformer for probabilistic next-return modeling on one-hour foreign-exchange bars. VAIOM separates input representation from output likelihood: continuous multivariate financial-event vectors preserve numerical structure at the input, while a categorical distribution over the next volatility-normalized return bucket supports cross-entropy training and likelihood evaluation. The selected 0.9M Hybrid Continuous Input model combines continuous event features with categorical asset metadata, a Mixture-of-Market-States return head, Gap, volatility-regime, and Ordinal auxiliary objectives, and full-sequence supervision. Models and preprocessing are fit using pre-2024 Train data; models are selected on 2024H2 Validation and evaluated without refitting on two 2025 Test periods. Across three independent training seeds, every model outperforms fixed single-bar LightGBM baseline in both Test halves. For the canonical checkpoint, paired gains over LightGBM are 0.029 and 0.043 bits per event. Validation experiments show that continuous input improves over discrete-token input under the same categorical return objective, full-sequence supervision improves over last-position training, and auxiliary representation shaping together with a mixture-structured return head improves return likelihood in controlled comparisons. A supporting capacity study finds that the smallest evaluated complete architecture rung achieves the strongest Validation likelihood on the present corpus.
Accurate stock price forecasting remains a challenging problem due to the noisy, nonlinear, and non-stationary characteristics of financial time series. Although recent advances in deep learning have improved predictive capabilities, most prior studies evaluate forecasting models primarily using statistical error metrics, with limited consideration of their practical value in trading and investment decision-making. This creates a gap between predictive performance and real-world economic utility. This study proposes a decision-oriented evaluation framework for multi-step stock price forecasting that jointly assesses predictive accuracy and trading profitability within a unified experimental setting. Using data from 91 ASX 100 stocks after data cleaning, with a testing period spanning 2019–2020, several deep learning architectures, including Multi-Layer Perceptron (MLP), Gated Recurrent Unit (GRU), Seq2Seq, and attention-based sequence models, are systematically compared under identical training and trading conditions. The results show that the Seq2Seq model achieved the best overall performance, obtaining the lowest average MAPE of 0.0293 and the highest ROI of 23.2%, while the attention-based model achieved a similar MAPE of 0.0294 but a lower ROI of 12.4%. Although differences in forecasting accuracy were relatively small, the Seq2Seq model achieved the highest observed trading profitability and generated a higher observed return than a passive market benchmark under the proposed evaluation framework. These findings suggest that evaluation based solely on prediction accuracy may not fully capture the practical value of forecasting models.
