The results provide significant, multi-perspective evidence that social effects of data could be a causal driver of LLM capability decay in continual pre-training, thereby motivating routine"cognitive health checks" for deployed and evolving LLMs.
Shuo Xing, Junyuan Hong, Yi-Fan Wang et al.· arXiv.org· 3 citations
While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions. In this paper, we take a first step toward systematically studying mathematical understanding in LLM...
Shuo Xing, Zi-Lin Dai, Cheng-Yuan Qian et al.· 0 citations
Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting mechanism to the client-trained, server-aggregated loop of federated learning, which holds back new learning to preserve earlier knowledge and burdens resource-constrain...