The article starts with the Transformer, which connects words in context, and pre-training, where models learn to predict text, and explains how fine-tuning adapts models to requests, with reinforcement learning from human feedback using human answer rankings.
The largest open German text datasets are built from broad web crawls, which provide scale but limited coverage of specific domains. We introduce RefinedGAW, a processed and annotated resource derived from 26 semi-annual crawls of the German Academic Web, totaling 122 million documents and 314 billion tokens, with full...
Maximilian Idahl, Oleh Astappiev, Benedikt Droste et al.· 0 citations
Estimating the correctness of large language model (LLM) predictions without access to ground truth is critical for reliable deployment in high-stakes settings. Existing approaches, such as entropy and perplexity, operate solely on final output statistics, discarding the rich information encoded in intermediate reasoni...
Chenyang Yu, Zifan Zeng, Qunli Zhang et al.· 0 citations
A long-lived assistant that brings up memories without being asked must decide at every turn whether anything in its store is worth an interruption. We show that this decision is a multiple-testing problem whose difficulty grows with the store size N. On replays of LongMemEval, a similarity threshold tuned to 5% false...
Yi-Chuan Chen, Xin Wang· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
The Voynich manuscript is a fifteenth-century book in a script that no one can read. One account holds that its text means nothing and that a scribe made it by copying his own words and changing them a little. The other holds that a real language lies hidden behind a cipher. We write each account as a program that fill...
In formal systems such as Lean 4, efficiently finding answers to mathematical problems and generating verifiable proofs is an important task in automated mathematical reasoning. In the traditional way where an upstream solver provides multiple candidate answers, the common workflow requires searching for proofs for eac...
Pengzhen Lin, Ke Zhang, Maziar Raissi et al.· 0 citations
Understanding how individual token choices affect reasoning length is important for characterizing reasoning models. Building on prior findings that changing the first reasoning token (hereafter, the first token) can substantially alter reasoning length, we focus on this initial choice to investigate how its influence...
Ruochen Liu, Hao Yan, Ziluowen Luo et al.· 0 citations
Structured semantic parsing is typically judged by whether the correct category is assigned, which leaves value recovery and instance-level associations unexamined: a system may recognize an event yet bind an argument to the wrong instance, or identify an intent yet supply an incorrect slot value. We present USP, a cro...
Zhongqiu Li, Boyang Wang, Ruiyu Fang et al.· 0 citations
Large reasoning models are costly to serve because they are large and because they generate long chains of thought (CoT). Weight pruning and CoT compression address these two costs separately, but they interfere in that pruning adds redundant reasoning on problems the model still solves and incomplete reasoning on thos...
Tao Jing, Chen Kang, Feifei Zhai et al.· 0 citations
Extracting salient information from long-horizon dialogue histories remains a significant challenge for Large Language Models (LLMs) due to the restricted context windows and efficient retrieval challenges. Current methods typically rely on history organization or local token-level optimization, suffering from informat...
Lian Yan, Zhen-Bo Xia, Hao-Tian Wang et al.· 0 citations
The stance expressed in responses from retrieval-augmented generation systems is significantly influenced by the stance distribution of the retrieved materials. Existing research on knowledge poisoning has primarily targeted factual errors. In contrast, this study focuses on injection attacks in which the facts are cor...
Pu-Yuan Yang, Shao-Xuan Dong, Liu-Jie Wu et al.· 0 citations
A detailed classification of LLMs is showcased, starting with the earlier models like BERT and leading to LLMs like GPT, Gemini, Claude and Grok, revealing a clear pattern: models that optimize the ratio between architectural complexity and data quality exhibit stronger reasoning stability and better alignment, without...
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.