Loss spikes in language model pretraining can waste substantial compute and derail otherwise healthy training runs. A primary driver of these instabilities is the growth and divergence of output logits in the final language modeling (LM) head. In this work, we study a particularly costly form of this instability associ...
Zaiqian Chen, Aditya Somasundaram, Sophie Li et al.· 0 citations
Text summarization is an important task for advancing natural language processing (NLP) research and applications in Indic languages. Despite progress in extractive and abstractive summarization, both monolingual summarization (MS) and cross-lingual summarization (CLS) remain relatively underexplored for these language...
Music audio generation spans instrumental clips, full songs with vocals, edits of existing recordings, and systems that play alongside a musician in real time. This monograph explains the methods behind these systems for readers familiar with neural-network training who are new to music generation. It introduces neural...
M. Modrzejewski, Joanna Kurkowska, Dorien Herremans· 0 citations
Existing code reveals past implementation decisions but rarely explains the reasoning behind them. When coding agents implement new features or modify existing functionality, past decisions may conflict with new task instructions. This raises a fundamental question for agentic software evolution: when multiple implemen...
Haichao Zhu, Geyang Xu, Zhaorui Yang et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Current datasets for text-to-video generation are largely limited to short clips depicting a single event or coarse-grained captions. As a result, they provide limited supervision for modeling long-take temporal dependencies, often leading to temporally inconsistent generations when describing multiple events. In this...
Pei-Yuan Zhu, Shao-An Xie, Yi-Fan Shen et al.· 0 citations
Large language models are increasingly used for financial guidance, yet little is known about whether their advice changes across social identities in Global South contexts. We introduce the Indian Financial Automated Advice Bias (IFAAB) benchmark, a counterfactual audit dataset of 31,734 GPT-4o-generated financial-adv...
Robot imitation learning uses demonstrations from operators of varying skill, but selecting training data remains challenging. Existing methods score demonstrations using smoothness, diversity, or estimated influence on policy performance, but differences in selection and evaluation protocols make it difficult to deter...
Coding agents are typically provided end-to-end behavioral specifications that the code they produce should satisfy, such as tests and input--output examples. How many such specifications must a developer provide, and which ones, for the agent to produce the developer's intended program? We argue that this cannot be de...
Aditya Golatkar, Alessandro Achille, Renos Zabounidis et al.· 0 citations
This work is the first to formalize Role Grounding Attack (RGA), an attack model that exploits a vulnerability in role-to-principal attribution through nested conversation structures and achieves high attack success rates.
Hao-Ran Bu, Yi-Da Shen, Li-Tian Zhang et al.· 0 citations
Fast inference and strict constraint satisfaction are essential for learning-based decision-making. Existing methods often rely on constraint-specific projections or feasible-set parameterizations, while iterative repair can be costly. We introduce Convex-SKM-Net, a convex constraint-satisfaction layer that forms linea...
EasyProver is presented, a Lean proving system built from recursive agents with isolated contexts, asynchronous search, parallel tactic probing, and an elastic elaboration service, and Aegis is introduced, which converts competitive programming problems into much harder verification tasks over real implementations.
Debargha Ganguly, Divya Raghunathan, Dejan Jovanovic et al.· 0 citations
When exact probabilities cannot be computed, \emph{density ratio estimation (DRE)} is a key subroutine in the wide range of applications where density ratios appear, from experimental design to generative modelling and reinforcement learning. In this paper, we introduce the concept of an \emph{expected log density rati...
Avi Amalanshu, Chandler Squires, Yizhou Lu et al.· 0 citations