The nature of improvement in each case is characterized: while recoverable concepts can be articulated through definitions and explanations, irrecoverable concepts cannot be transmitted, even with examples, and might require novel learning mechanisms.
Alexander Spangher, Daniel E. Ho, Di-Yi Yang et al.· 0 citations
Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs. We propose \textbf{C}ontinuous \textbf{L}at\textbf{E}nt \textbf{A}dapter \textbf{R}outing (CLEAR), a conditional safety adaptation f...
Chengxiao Wang, Enyi Jiang, Xiaojing Liao et al.· 0 citations
Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects of the evaluation conditions such as the scaffolds or tasks. This makes reliability claim-dependent: an evaluation that reliably ranks deployed systems may not reliably...
Michael Hardy, Ruhana Azam, Anka Reuel et al.· 1 citation
The paper concludes that inference-time interventions are inadequate for combating the Artificial Hivemind, and shows that more generalizable solutions are needed at the model training level, and that an inference-time intervention (prompting) reliably raises measured response diversity.
Rylan Schaeffer, Brando Miranda, Joshua Kazdan et al.· 0 citations
This work introduces HeadEntropy, a method that predicts answer correctness by measuring how susceptible each attention head's pattern is to modification under further gradient updates, and evaluates broadly across 5 instruction-tuned LLMs and 5 datasets spanning general knowledge, multi-hop reasoning, mathematical rea...
S. Ostmeier, Brian Axelrod, M. Varma et al.· 5 citations
This work uses text optimization to detect subliminal learning effects and describes them as legible prompts and presents SALVE (Search-Aided Latent Verbalization) as a method to proactively detect subliminal learning effects.
Nathan Hu, Sanmi Koyejo, Christopher Potts· 1 citation
This work adapt convergent and discriminant validity from the social sciences into an approach for interrogating AI benchmarks, applying it to 56 capability and safety benchmarks across 53 models and asking analogous questions at the item level using item response theory (IRT) models.
Meera Desai, Sang T. Truong, Hanna Wallach et al.· 0 citations
This work audits ChatGPT, Claude, and Gemini across seven systems and nine benchmarks spanning general capability, social bias, and sycophancy, finding systematic API--interface differences in both accuracy and consistency.
Jennifer Wang, Joachim Baumann, Daniel E. Ho et al.· 0 citations
This work proposes PU-DPO, a preference optimization framework to prevent omission noise from corrupting the preference signal, and reformulates the objective under a positive-unlabeled (PU) learning framework, treating absent mentions as unlabeled rather than truly negative.
Y. Kobayashi, P. Ramesh, Muhammad Ahmed Chaudhry et al.· 0 citations
It is found that the model's coding proficiency is inversely related to predictive performance in this setting, with the LLM better understood as a generative solver conditioned on context rather than a faithful predictor of solver behavior.
Fagun Patel, Sang T. Truong, Duc Q. Nguyen et al.· 0 citations
This report argues that affirmative AI coverage with limits in the billions is achievable by 2030, but only with industry-wide coordination, and lays out an eight-component AI insurance stack spanning incident data collection, catastrophe modeling, standards, contract design, risk selection, pricing, monitoring, and cl...
Cristian Trout, Sanmi Koyejo, S. Romanosky et al.· arXiv.org· 0 citations
This work argues for a hybrid public-private approach: keeping the benchmark content private to maintain integrity while making the benchmark design public to build trust, which offers a path toward trustworthy measurement of ever-growing data-intensive AI systems.
Sang T. Truong, S. Wang, Angelina Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.