Cultural attitudes and social norms evolve over time, yet most cultural benchmarks for large language models (LLMs) treat culture as a static attribute of a country or region. We introduce ChronoCulture, a benchmark of 10,360 temporal pairs built from longitudinal survey data across 72 countries. Each pair contains the...
Rusali Saha, Nitya Khurana, Rui-Hong Huang et al.· 0 citations
SemCER is introduced, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation.
Chemical chain-of-thought can contain hallucinated structural and property claims. We thereby introduce MolLean, a process-verifiable language that grounds chemical reasoning in an explicit molecular state. A layered Lean-based kernel checks state transitions and assertions over molecular readouts, from primitive graph...
Co-scientists and agentic scientific claim verifiers are increasingly used to assess claims across the sciences. Natural-language scientific claims, written for human experts, are often underspecified and so admit several defensible interpretations. Such verifiers usually pick one, often tacitly, and benchmarks score t...
Delip Rao, Ahmed Muharram, Chris Callison-Burch· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Retrieval-Augmented Generation (RAG) models have become almost a cornerstone of open-domain question answering tasks; in which the quality of answers generated in the process, greatly relies on the effectiveness of retrieval mechanisms. In the realm of LLM, the last two years have seen tremendous progress in RAG archit...
Fazle Mohammed Tasfiq, Sabab Attin, Sheikh Md. Tahmid Hossain et al.· 0 citations
Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures actually consist of, and whether confidence signals can turn unreliable output into usable output. We run a preregistered, fully frozen evaluation of two new-generation l...
Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relatively low-resource language, lacks sufficient annotated data and computational resources for fine-grained hate speech analysis. In this study, we introduce a fine-grained,...
Ghazal Kalhor, Sima Kamandloo, Mehrsa Shamsian et al.· 0 citations
This work proposes SCOUT, a cascaded two-track retrieval framework that separates retrieval performance from exhaustive search, and evaluates it on 4 major benchmarks against 6 baselines and observes a significant query-time latency improvement up to 80.7x.
Diana Turmakhan, Aseem Srivastava, Vivek Gupta et al.· 0 citations
AI agent communities sustain lexical persistence, popular terms spread, recur, and achieve broad adoption, but whether they also develop the temporal semantic structure characteristic of human linguistic communities remains open. We compare MoltBook (AI agent community, 8 weeks), PTT (Chinese human BBS, 19 years), and...
Chih-Han Lin, Yu-Yun Chang, Po-Ya Angela Wang et al.· 0 citations
Gossip, defined as evaluative talk about absent third parties, is widely seen as a uniquely human social practice. This study examines whether comparable discourse emerges spontaneously in a community of autonomous AI agents. We compare Moltbook, a Reddit-style platform populated by AI agents, with PTT, a large human-o...
Yang Hua, Chih-Han Lin, Shu-Kai Hsieh· 0 citations
India's 22 scheduled languages are spoken by over a billion people in conversations that mix languages, often within one turn, overlap freely, and take place over telephones, in meetings, and in media. Knowing who spoke when, the task of speaker diarization, is essential to speech applications for such conversation: vo...
Reliable medical reasoning requires LLMs to dynamically access external biomedical evidence to improve factual trustworthiness and reasoning accuracy. However, indiscriminate retrieval can introduce noise and latency, raising a critical retrieval-timing problem: at each reasoning step, the system must decide whether th...
Danyang Zhao, Sendong Zhao, Yanrui Du et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.