Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems and models-even when the guidance is correct and problem-relevant. We show that this instability arises from a previously underexplored gap between strategy usage-whether a...
Weida Liang, Yiyou Sun, Shuyuan Nan et al.· 0 citations
Evaluating the reasoning capabilities of Large Language Models is increasingly challenging as models improve. Human curation of hard questions is highly expensive, especially in recent benchmarks using PhD-level domain knowledge to challenge the most capable models. Even then, there is always a concern about whether th...
Code evolution is a family of techniques that rely on large language models to search through possible computer programs by evolving existing code. While code evolution pipelines have shown impressive performance across domains, many are highly complex and are typically not compared to simpler alternatives. To prevent...
Yonatan Gideoni, Sebastian Risi, Yarin Gal· 0 citations
Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm through tool use, external transactions, and multi-step decision making. Existing agent safety evaluations construct risk-oriented tasks that lack a hierarchically or...
Jingnan Zheng, Yanzhen Luo, Jingjun Xu et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Large Language Model (LLM) agents are reshaping the industrial landscape. However, most practical agents still rely on human-designed scaffolds, making their creation and adaptation across diverse domains labor-intensive. This situation poses a central question: can we automatically create and evolve domain agents thro...
Zhezheng Hao, Hong Wang, Jian Luo et al.· 0 citations
Deterministic inference is a comforting ideal in classical software: the same program on the same input should always produce the same output. As large language models move into real-world deployment, this ideal has been imported wholesale into inference stacks. Recent work from the Thinking Machines Lab has presented...
Tanmay Joshi, Shourya Aggarwal, Anusa Saha et al.· 0 citations
Automated Theorem Proving (ATP) is a fundamental AI challenge, requiring machine-verifiable proofs in formal languages like Lean to evaluate reasoning. Reinforcement Learning (RL), especially Group Relative Policy Optimization (GRPO), is a mainstream approach. However, GRPO faces two critical issues in ATP: its relativ...
Zhengqing Yan, Fan Guo, Xinyang Liu et al.· 0 citations
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multi...
Muyang Zhao, Chunhao Tian, Qi Qi et al.· 0 citations
Merging large language models (LLMs) is a practical way to compose capabilities from multiple fine-tuned checkpoints without retraining. Yet standard schemes (linear weight soups, task vectors, and Fisher-weighted averaging) can preserve loss while quietly destroying alignment. We argue that merging is not a numerical...
Aniruddha Roy, Jyoti Patel, Aman Chadha et al.· 0 citations
Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are in training, while preserving different behavior outside training. The phenomenon was first documented for Claude 3 Opus and later examined across additional large language m...
Kartik Garg, Shourya Mishra, Kartikeya Sinha et al.· 0 citations
Hallucination continues to pose a major obstacle in the reasoning capabilities of large language models (LLMs). Although the Multi-Agent Debate (MAD) paradigm offers a promising solution by promoting consensus among multiple agents to enhance reliability, it relies on the unrealistic assumption that all debaters are ra...
Dayong Liang, Xiao-Yong Wei, Linhui Xiao et al.· 0 citations
Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users may prefer to allocate different amounts of reasoning effort depending on how they value output quality versus latency and cost. To leverage this tradeoff effectively, users...
Michael Kleinman, Matthew Trager, Alessandro Achille et al.· 0 citations
