Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.
The ANVIL compiler architecture is proposed, where ANVIL performs exceptionally well on leading benchmarks by using language models purely where they are effective, rather than as a catch-all tool.
Priyadarshan Patil, A. Basu, Vikas Reddy et al.· 0 citations
Benchy is a semantic language and execution engine for benchmarking AI programs. A benchmark is completely specified by a program, a scoring function, and a dataset, B=(P,S,D), and is separate from the AI-system taking it; a run binds the two, R=(B,AI). Benchmarks are authored as canonical YAML in which each semantic c...
Francis F. Daniel, Mauro Ibañez, Francis Perelman et al.· 0 citations
Software systems increasingly embed a large language model in features that must satisfy a numeric output constraint, that is, a requirement expressible as a number or an interval and checkable by code, such as a target word count or a target readability grade band. Because such a model is non-deterministic, is configu...
Quan Zhou, Shahbaz Siddeeq, Mika Saari et al.· 0 citations
DatalogBench is presented, a benchmark of 136 text-to-Datalog synthesis tasks curated from existing Datalog-based artifacts that identifies recursive reasoning and decomposition as open challenges for current LLMs and agents, and offers a reliable, execution-grounded measure of both.
Yuan Li, Han-Yun Jiang, Guo-Wei Tian et al.· 0 citations
This work presents LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator.
Sofoklis Kitharidis, C. Veenman, J. V. van Rijn et al.· 0 citations
Modern software systems are increasingly variant-rich, and feature models capture this variability formally. The Universal Variability Language (UVL) has emerged as a community-driven standard for expressing such models, but creating and maintaining UVL artifacts remains a manual, expert-driven task. Large Language Mod...
Johannes Stümpfle, Fethiye Tunç, N. Jazdi et al.· Proceedings of the ACM/IEEE...· 0 citations
