A common approach to question answering with retrieval-augmented generation (RAG) is to concatenate documents into a single context and pass it to a language model to generate an answer. While simple, this strategy can obscure the contribution of individual documents, making attribution difficult and contributing to th...
Jinghong Chen, Jingbiao Mei, Guangyu Yang et al.· 0 citations
Results demonstrate how LLM semantic similarity scores are sensitive to the aforementioned parameters in ways that go beyond the semantic change itself; this framework offers a practical toolkit for auditing and comparing scoring behavior across current and future models.
Sinan G. Aksoy, Alexandra A. Sabrio, Erik Vonkaenel et al.· arXiv.org· 2 citations
OmniBehavior is introduced, the first predictive user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework.
Jiawei Chen, Ruo-Xi Xu, Boxi Cao et al.· arXiv.org· 12 citations· ⚡1
The extension of context windows in Large Language Models is typically facilitated by scaling positional encodings followed by Continued Pre-Training (CPT). While effective, this paradigm is notoriously data-hungry and computationally expensive, requiring massive long-text corpora to recalibrate the model to the shifte...
Ning Yang, Hengyu Zhong, Wentao Wang et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
ActTail is proposed, a TopK magnitude-based activation sparsity method with global activation sparsity allocation grounded in Heavy-Tailed Self-Regularization (HT-SR) theory that improves both perplexity and downstream task performance at high sparsity compared to uniform allocation.
How much of zero-shot benchmark accuracy is captured by word-level alignment between a pre-training corpus and the benchmark target? We study this question through word-level unigram cross-entropy (UCE), a tokenizer-agnostic, model-free measure of lexical alignment between a reference corpus and a target benchmark. Lik...
Jeonghoon Kim, Woojin Chung, Woomin Song et al.· 0 citations
The advancement of Large Language Models (LLMs) has catalyzed the development of autonomous agents capable of navigating complex environments. However, existing evaluations primarily adopt a deductive paradigm, where agents execute tasks based on explicitly provided rules and static goals, often within limited planning...
Fangzhi Xu, Hang Yan, Qiushi Sun et al.· 0 citations
Complex question answering across text, tables and images requires integrating diverse information sources. A framework supporting specialized processing with coordination and interpretability is needed. We introduce DeALOG, a decentralized multi-agent framework for multimodal question answering. It uses specialized ag...
Enhancing the moral alignment of Large Language Models (LLMs) is a critical challenge in AI safety. Current alignment techniques often act as superficial guardrails, leaving the intrinsic moral representations of LLMs largely untouched. In this paper, we bridge this gap by leveraging Moral Foundations Theory (MFT) to m...
Luoming Hu, Jingjie Zeng, Liang Yang et al.· 0 citations
Singlish is a creole rooted in Singapore's multilingual environment that continues to evolve alongside social and technological change. We examine diachronic stylistic change across a decade of informal digital messages and ask whether Large Language Models (LLMs) can generate temporally neutral outputs approximating t...
Linus Tze En Foo, Weihan Angela Ng, Wenkai Li et al.· 0 citations
Despite continuous advances in medical technology, the global distribution of health care resources remains uneven. The development of large language models (LLMs) has transformed the landscape of medicine and holds promise for improving health care quality and expanding access to medical information globally. However,...
Rui Yang, Huitao Li, Weihao Xuan et al.· 0 citations
Latent tokens are gaining attention for enhancing reasoning in large language models (LLMs), yet their internal mechanisms remain unclear. This paper examines the problem from a reliability perspective, uncovering fundamental weaknesses: latent tokens function as uninterpretable placeholders rather than encoding faithf...
Yuyi Zhang, Boyu Tang, Tianjie Ju et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
