Category
9,568 papers
Repository-level coding benchmarks largely assume that tasks and environments remain unchanged throughout execution. In software development, code, dependencies, runtime state, and requirements can change while an agent is still working. We introduce SWE-Shift, a benchmark for evaluating coding agents under such mid-ex...
Large language models (LLMs) are increasingly used to support decisions under incomplete information, where deciding what to look up and when to stop matters as much as the final answer. However, LLMs fall into an information trap: They pay for checks, i.e., costly pieces of evidence, that cannot change their decision...
This work develops a general framework that connects data-driven behavioural discovery with analysis and experimental investigation for trajectories of varying length, event order and agent roles, and identifies advances in coding, analysis and experimentation needed to support its wider use.
LLM agents are increasingly deployed in collaborative settings where they pursue separate tasks while sharing resources and operating under review or oversight processes. Such settings create safety risks not only when agents are malicious, but also when they are simply trying to help. Prior work has largely studied ad...
Reach audiences
Advertise in front of researchers, engineers, and readers.
This work introduces a repeated, multi-party, multi-issue negotiation framework in which recurring agents participate in a sequence of negotiation games while one participant covertly attempts to prevent agreement.
Large language models (LLMs) often generate more reasoning tokens than needed. Limiting the reasoning budget reduces cost, but forcing an answer from an incomplete trace can hurt accuracy. System-1 decision models select among candidate options without producing a reasoning response. We evaluate Jev as a decision reade...
"Abliterated" ("uncensored") builds of open-weight language models remove the activation direction that mediates refusal, and are popular partly on the assumption that a model that no longer refuses is free of political censorship. We test this on two matched pairs of 2-bit models: NVIDIA's Nemotron 3.5 Bolt 74B-A9B an...
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global description. We propose Observation-Adaptive Video Captioner (OACap), a caption-centered agent that turns unsupported or missing i...
Spatial intelligence requires agents to actively acquire visual evidence and integrate observations into a coherent representation of the three-dimensional world. Yet spatial question answering offers limited evidence of whether models can construct accurate metric maps through autonomous exploration. We introduce Spat...
This work introduces $\mathrm{ASR}_{\mathrm{STEP}}$, which penalizes additional victim-model calls while retaining binary success judgments, and considers GCG, PAIR with a step budget, and a single-step PAP baseline on JailBreakBench across three model families.
Pluralistic alignment aims to represent meaningful variation in human preferences rather than collapse heterogeneous judgments into a single target. Yet existing work often mixes three distinct questions: what pluralistic behavior a model should exhibit, whose preferences define the reference, and what evidence is suff...
Unified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. However, each capability is evaluated in isolation on specialized benchmarks and a basic question remains unanswered: can these models understand the audio they generate/edit i.e. do they ex...
Natural Language Processing
Our most capable vision-language-action model enhances robot dexterity, reliability, and safety
A Blog post by Ai2 on Hugging Face
A Blog post by IBM Research on Hugging Face
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
