Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level. We introduce CultureLens, a reference-grounded benchmark and mechanistic study of how LLMs use cultural information. CultureLens contains 9,261 scenarios...
Samee Arif, F. Danışman, Angana Borah et al.· 0 citations
It is found that curiosity-oriented questions remain rare (17-28% of all questions), but are associated with more exploratory and epistemically rich subsequent user responses, with varying patterns across datasets.
Angana Borah, Rachana Ramchandar, I. Nair et al.· 0 citations
SWE-Verifier is introduced, a lightweight generative model that selectively triggers candidate sampling and verification that achieves comparable or higher resolution than the trajectory and all-step baselines at substantially lower cost.
Kai-Jian Zou, V. Chennareddy, Yun-Xiang Zhang et al.· 0 citations
Exposure to misaligned advisors shifted preferences away from optimal options and toward inferior alternatives, increasing the odds of preferring the incentivized option over the optimal option by ≈5 to 8 times (up to +38 percentage points).
Sahand Sabour, June M. Liu, Siyang Liu et al.· Proceedings of the National...· 3 citations· ⚡1
Step-level reasoning evaluators are commonly based on autoregressive language models, whose causal attention restricts each step representation to the problem, previous steps, and the current step. Yet, when the complete solution is available, the validity of an earlier step may become clearer only through its downstre...
Yi-Ming Feng, Nai-Hao Deng, Yu-Long Chen et al.· 0 citations
It is revealed that the energy divide persists across models, hardware, and tasks, suggesting a systemic energy inequity in multilingual LLM deployment and is recommended that the community treat energy as a first-class evaluation axis, extend reporting checklists and model cards to include it, and adopt deployment-sid...
It is argued that BBQ-style multiple-choice abstention benchmarks measure a single structural cue, and a model that solves them does not thereby become fair, and is called for evaluation suites that cover a broader spectrum of fairness alignment.
Nai-Hao Deng, Samee Arif, Shuai-Chen Chang et al.· Proceedings of the 2026 Conf...· 0 citations
KIDBench, a benchmark for evaluating child-facing LLM safety for ages 7-11 using a LLM-as-a-Judge rubric grounded in developmental-psychology, is introduced and KIDGuardLlama, a child-safety evaluator, and KIDLlama, a child-safe response model are introduced.
Samee Arif, Angana Borah, Rada Mihalcea· arXiv.org· 2 citations
AtlasNLP is introduced, a country-aware atlas of over 13,000 NLP dataset records across normalized NLP task categories, tracking both the populations represented and where datasets are produced, showing that dataset coverage is highly uneven across countries and tasks and language coverage does not imply geographic rep...
Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.