BigO(Bench) is introduced, a novel coding benchmark designed to evaluate the capabilities of generative language models in understanding and generating code with specified time and space complexities, and results from evaluating multiple state-of-the-art language models on this benchmark are presented.
Pierre Chambon, Baptiste Rozière, Benoît Sagot et al.· 11 citations
These token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking, with this role-level decomposition most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit lo...
Wissam Antoun, Francis Kulumba, Théo Lasnier et al.· 1 citation
This work makes execution time learnable through three stages: how code is tested, by building DMC-Optim with large optimization tests and a calibrated sandbox; how speed is turned into reward, by composing correctness and speed in the RL environment and using an offline simulator to predict the most promising configur...
Pierre Chambon, Kun-Hao Zheng, Juliette Decugis et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.