Two-Stage Robust Optimization (2RO) with discrete uncertainty is challenging, often rendering exact solutions prohibitive. Scenario reduction alleviates this issue by selecting a small, representative subset of scenarios to enable tractable computation. However, existing methods are largely problem-agnostic, operating...
Tianjue Lin, Jianan Zhou, Jieyi Bi et al.· 0 citations
LLM-as-a-Judge pipelines have become the de facto evaluator for agent safety, yet existing benchmarks treat their verdicts as ground-truth proxies without checking whether the verdicts depend on the agent's behavior or merely on how the evaluation policy happens to be worded. We argue that any trustworthy safety judge...
We introduce the Architecturally Admissible Compressor (AAC), which adapts landmark-based A* heuristics to query demand under a fixed label-memory budget. AAC provides two continuous relaxations of ALT (A*, Landmarks, and Triangle inequality) selection. With exact landmark distances, every parameter value defines a low...
LLM agents are rapidly becoming the practical interface for task automation, yet the ecosystem lacks a principled way to choose among an exploding space of deployable configurations. Existing LLM leaderboards and tool/agent benchmarks evaluate components in isolation and remain fragmented across tasks, metrics, and can...
Yunxiao Shi, Wujiang Xu, Tingwei Chen et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Next-generation AI must manage vast personal data, diverse tools, and multi-step reasoning, yet most benchmarks remain context-free and single-turn. We present ASTRA-bench (Assistant Skills in Tool-use, Reasoning \& Action-planning), a benchmark that uniquely unifies time-evolving personal context with an interactive t...
Zidi Xiu, David Q. Sun, Kevin Cheng et al.· 0 citations
Prompt design affects the moral judgments and refusal behavior produced by large language models (LLMs), yet controlled comparisons remain fragmented across datasets and models. We introduce ProMoral-Bench, a benchmark-style comparison of 10 prompting paradigms across four LLM families. Using ETHICS, Scruples, WildJail...
Rohan Subramanian Thomas, Shikhar Shiromani, Abdullah Chaudhry et al.· 0 citations
LLM routing aims to reduce inference costs by assigning easy queries to less expensive models and reserving stronger models for cases in which they are needed. However, we find that existing routers often violate this principle under loose cost budgets: as the budget increases, they increasingly select the strongest an...
As Large Language Models (LLMs) move from curated training sets into open-ended real-world environments, a fundamental limitation emerges: static training cannot keep pace with continual deployment environment change. Scaling training-time and inference-time compute improves static capability but does not close this tr...
Minhua Lin, Hanqing Lu, Zhan Shi et al.· 0 citations
A modular framework of depth-recurrent attention mixtures (Dreamer), combining sequence attention, depth attention, and sparse expert attention is introduced, which alleviates the hidden-size bottleneck through attention along depth, decouples scaling dimensions, and allows depth-recurrent models to scale efficiently a...
Jonas Knupp, J. Metzen, Jeremias Bohn et al.· 15 citations
A shared vocabulary between cognitive science and LLM research is established and a fine-grained evaluation framework is introduced, enabling diagnosis of reasoning failures and principled development of models that reason through robust cognitive mechanisms rather than spurious shortcuts, while providing tools to test...
S. Li, Priyanka Kargupta, Hao-Cheng Wang et al.· arXiv.org· 26 citations· ⚡2
As vision-language models (VLMs) gain prominence, their multimodal interfaces also introduce new safety vulnerabilities, making the safety evaluation challenging and critical. Existing red-teaming efforts are either restricted to a narrow set of adversarial patterns or depend heavily on manual engineering, lacking scal...
Zhaorun Chen, Xun Liu, Mintong Kang et al.· 0 citations
In this paper, we present SOCIA-$\nabla$, a diagnosis-driven multi-agent framework for automated simulator-code construction. SOCIA-$\nabla$ iteratively refines executable simulator code within a textual computation graph, where LLM-driven agents synthesize, execute, evaluate, diagnose, and repair simulators. A grounde...
Yuncheng Hua, Sion Weatherhead, Mehdi Jafari et al.· 0 citations