It is found that, once prefill is measured without prefix caching, the recurrent architectures pay no prefill penalty: their linear-time mixing undercuts quadratic attention on long prompts, so their efficient decode adds to the prefill saving and lowers total request energy from the first output token at large batch.
Bo-Le Ma, Ayesha Afzal, Jan Eitzinger et al.· arXiv.org· 3 citations· ⚡1
DUET budget-aware control strategies are valuable not only for accelerating training, but also for improving the quality of the learning signal, contrary to the usual pattern in which efficient methods trade off quality as compute decreases.
Reasoning distillation aims to transfer multi-step reasoning capabilities from large language models to smaller, efficient ones. While recent methods have shown promising gains, they typically rely on static teacher-student hierarchies and frame distillation as trajectory imitation. This is misaligned with the structur...
Low-Rank Adaptation (LoRA) is proposed as a structural regularizer for critic learning from scratch that reduces critic loss during training and improves overall policy performance, achieving the best or competitive results on most tasks.
Yuan Zhuang, Yuexin Bian, Si-Hong He et al.· arXiv.org· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Neural routing solvers rarely stop at their first solution: constructed tours are usually refined by search, such as sampling or local search. Neural improvement methods learn this search as a policy that repeatedly applies local moves, but existing policies degrade sharply beyond their training size. We argue that siz...
Genotype Bidirectional Encoder Representations from Transformers model (GenoBERT), a transformer-based, reference-free framework that tokenizes phased genotypes and uses self-attention mechanism to capture both short- and long-range linkage disequilibrium (LD) dependencies, is presented.
Lei Huang, Chuan Qiu, Kuan-Jui Su et al.· arXiv.org· 1 citation
This work presents a metric-level study of LLM-mediated search policy learning, studying how LLMs construct and adapt exploration-exploitation strategies under multiple operational definitions of exploration, including informativeness, diversity, and representativeness.
Andrea Carbonati, Mohammadsina Almasi, H. Anahideh· arXiv.org· 1 citation
It is shown that the transition from productive to redundant reasoning is decodable from late-layer hidden states and representations separate the two phases around first-correct-solution boundaries beyond what position and lexical controls explain, and distinguish correct from incorrect solution endings.
While Multimodal Large Language Models (MLLMs) show promising performance in automated electrocardiogram interpretation, it remains unclear whether they perform actual step-by-step reasoning or just rely on superficial visual cues. To investigate this, we introduce ECG-Reasoning-Benchmark, a novel multi-turn evaluation...
Jungwoo Oh, Hyunseung Chung, Junhee Lee et al.· 0 citations
State-of-the-art deep reinforcement learning (RL) methods have achieved remarkable performance in continuous control tasks, yet their computational complexity is often incompatible with the constraints of resource-limited hardware, due to their reliance on replay buffers, batch updates, and target networks. The emergin...
Riccardo De Monte, Matteo Cederle, Gian Antonio Susto· 0 citations
A Brouwer-inspired assertibility constraint for responsible AI is proposed: in high-stakes domains, systems may assert or deny claims only if they can provide a publicly inspectable and contestable certificate of entitlement; otherwise they must return Undetermined.
Fine-tuning an LLM to maximize performance on a downstream task requires finding the optimal data and model configuration. This is a black-box optimization problem that Bayesian optimization (BO) addresses by sequentially evaluating training configurations and using observed performance feedback to adaptively guide the...
Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong et al.· 0 citations
Research Manager Mike Walker reflects on a career spanning hacker culture, cybersecurity, and AI, sharing lessons about trust, teamwork, leadership, and why solving difficult problems requires curiosity, judgment, agility—and an understanding of the rules. The post Learn the rules: The hacker mindset appeared first on Microsoft Research.
Learn how Google Antigravity and our Cheminformatics Skills accelerate drug discovery workflows, search compounds, and propose novel rigidified macrocycles.
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
MIT News · Artificial Intelligence· news.mit.eduOct 27, 2026
Prabha Jha used her experience with MIT Open Learning’s program for senior executives to develop AI solutions for her employer, Verizon, and to write a book.
