Large language models are deployed globally, yet safety alignment and evaluation remain concentrated in English: harmful requests refused in English can elicit unsafe completions when translated into low-resource languages. We evaluate ten widely deployed LLMs across 79 languages (English plus 78 low-resource) under a...
Jalil Alva, David Harper, Abigail E. Thornton et al.· 0 citations
This work evaluates compaction on later questions of language-model agents, using four corpora that annotate the evidence of several questions per text and recorded agent trajectories, and compares every method with random deletion at the same budget.
Jun-Wei Yao, Yong-Xin Zhou, Zi-Ke Yang et al.· 0 citations
Effective detection of hate content is essential for maintaining healthy online communities, yet performance can vary substantially across languages. This work investigates a core practical question: Can training data from one language support hate-speech detection in another language with comparable performance? To an...
Preference labels align model training, but the judgments behind them are difficult to inspect or verify. In this paper, we examine how preference construction can be checked when intermediate decisions are recorded, and which of them remain uncertain. We present Symbolic Preference Labeling, a method in which a pretra...
Chenhan Lyu, Amir Rahmani· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Hallucination stress tests can yield similar average scores across language models even when different prompts elicit hallucinations in each model. We develop a procedure that iteratively edits source-grounded seed prompts to expose hallucinations in a target model. An LLM editor revises each prompt in response to the...
Jiho Lee, Bingquan Zhang, Lin Ting Liang et al.· 0 citations
Medical workflows often require multiple interdependent analyses, but existing specialist and generalist AI models are not designed to coordinate them. We introduce MMedAgent-Plus, a multimodal medical tool-use agent that combines a vision-language planner with 18 leading specialist models spanning 12 tasks and 8 modal...
Ruoxin Wang, Yuanting Pan, Binxu Li et al.· 0 citations
Profile-DP is introduced, a policy-aware local differential privacy framework designed specifically for user-profile release in personalized LLM inference, which provably bounds the release of user-withheld facts while treating public instructions, profile schemas, and authorized facts as public side information.
Mingchen Li, Zi-Fan Peng, Ke-Wei Sha et al.· 0 citations
Web agents are typically evaluated on English-language interfaces, leaving their ability to operate across the world's multilingual websites largely unexplored. This gap is especially significant for low-resource languages, where standardized resources for evaluating web agents remain limited. To address this gap, we i...
Ocean Monjur, Daud Ahmad Nisar, Shahriar Kabir Nahin et al.· 0 citations
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors w...
Chenpan Li, Thies Brockmoeller, Stefanie Jegelka et al.· 0 citations
Data selection plays a central role in modern large language model training, as the quality of training data strongly influences downstream performance. A common approach is to curate a development set that encodes the desired skills (e.g., reasoning or multilinguality) and select data that are similar to them. Existin...
Retrieval-augmented QA can fail even when the correct answer is in the retrieved evidence. This can happen when a competing answer satisfies some, but not all, of the question's requirements and is mentioned frequently in the retrieved documents, which can make it appear to be the best-supported answer. To test whether...
Jennifer Hsia, Zachary C. Lipton, Aarti Singh· 0 citations
MEDIATOR is introduced, a discrete ontology-agnostic tokenizer that maps any clinical concept expressed as text into a shared vector-quantized codebook, turning EHR harmonization into open-vocabulary tokenization without manual mappings or a fixed vocabulary.
Tong Ding, Cai-Wei Tian, Dongmin Bang et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
