Think-like-LSTM: Memory-Augmented Large Language Models via Dynamic Fine-Tuning for Financial Risk Assessment

Abstract

Financial risk assessment (FRA) seeks to forecast the likelihood of user default based on transaction sequences, which are inherently sequential and enriched with textual attributes. Despite the remarkable potential of large language models (LLMs) in sequential modeling and semantic understanding, developing LLM-based FRA solutions still presents unique challenges due to the temporal dynamics of transaction patterns and the need to capture long-term dependencies across prolonged behavioral horizons. To tackle these issues, in this paper, we propose FraLLM, a novel LLM fine-tuning framework that seamlessly internalizes transaction-oriented knowledge for FRA. FraLLM is designed from an interesting idea of Think-like-LSTM, enabling dynamic-aware LLMs to reason over long-range transaction trajectories. Specifically, we propose a Dynamic Fine-tuning Paradigm, which reprograms the transaction sequences into timestamped text prototypes and fine-tunes LLMs via a next-text prediction task. It dynamically offers linguistic gradients across time steps, facilitating transaction dynamics. Meanwhile, we introduce the Memory Token Mechanism that recurrently aggregates historical text prototypes into a compact, continuously updated memory token. This token allows LLMs to effectively synthesize long-term transaction history while ensuring cost-efficiency. Extensive experiments on three real-world industrial transaction datasets show that FraLLM outperforms all baselines, achieving substantial gains in AUC (+11.2%), Recall@10 (+9.1%), and KS (+14.8%). Moreover, our proposed FraLLM yields significant improvements in online testing within the WeChat Pay financial system, demonstrating its effectiveness and application value for industrial deployment.