Skip to content

Right Answers, Costly Models: The Efficiency Gap in LLM-based Optimization Modeling

Sep 2026 · arXiv.org · Vol abs/2609.38884 · 1 citation · 55 references
Computer Science

TL;DR

This work systematically investigates whether LLMs can identify problem structure from natural-language descriptions and apply suitable optimization modeling techniques to generate mathematical models and solver code that solve the problems correctly and efficiently.

Abstract

Optimization modeling formulates real-world decision problems as mathematical programs that solvers can use to find optimal decisions. Large language models (LLMs) can automate this process, but the resulting correct formulations can require substantial time and memory to construct and solve, limiting practical scalability. Therefore, we systematically investigate whether LLMs can identify problem structure from natural-language descriptions and apply suitable optimization modeling techniques to generate mathematical models and solver code that solve the problems correctly and efficiently. To this end, we first curate OptTips, a knowledge base of 50 expert modeling techniques in eight families. Using this knowledge, we develop OptDachshund, a multi-agent framework that transforms problems from existing optimization benchmarks into new tasks for evaluating LLMs'use of modeling techniques. It constructs conventional and expert mathematical models with solver code for the same task and data, providing baselines for correctness and computational cost. The resulting EfficientOpt benchmark contains 561 expert-reviewed tasks with paired reference implementations. Evaluation of 11 representative LLMs reveals an efficiency gap on correctly solved tasks with comparable measurements: for every LLM, most generated programs take longer to solve than their expert counterparts. Within the comparable reference-size subset, 57\% of programs with correct objective values and fewer variables and linear constraints have longer recorded solver times. Case studies show that different modeling techniques can achieve the same optimal value at similar recorded cost. Faster solving may not reduce execution time if the code takes longer to prepare data and build the model. LLM optimization modeling should therefore be evaluated for both correctness and computational efficiency.

View source

Similar papers

Open access Sep 2026

Bench of Euler: A Benchmark for Evaluating the Problem-Solving Abilities of Large Language Models

Large language models (LLMs) have recently improved their problem-solving abilities and can solve complex mathematical problems with an increasing accuracy, necessitating the development of more challenging benchmarks. Over the years, the performance of LLMs on several benchmark datasets has also improved, motivating t...

Anurag Dutta, S. Priya, A. Ramamoorthy et al. · 0 citations

TACIT: Optimization Models that Learn from Their Mistakes

This work addresses the challenge of automatically repairing misspecified formulations using historical data consisting of past solutions and subsequent user overrides to combine the reasoning capabilities and prior knowledge of LLMs with the formal grounding provided by optimization.

Maxime Bouscary, Marco Molinaro, Si-Rui Li et al. · 0 citations
#artificial intelligence Preprint Sep 2026

OptiSkill: A Hierarchical and Evolving SkillBank for LLM-Based Optimization Modeling

Automated operations research (OR) modeling requires LLMs to translate natural-language decision problems into correct mathematical programs. Existing methods can improve individual formulations, but they often solve problems in isolation, retaining little reusable experience and repeating similar formulation errors. P...

Rui Zhao, Rui Liu, Yuan Zuo et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

This work introduces OR-Clarify, a benchmark for pre-formulation clarification and proposes Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop.

Si-Han Ge, Yi-Chen Lin, Chen-Yu Zhou et al. · 0 citations
Preprint Sep 2026

Online Automated Algorithm Design with Large Language Models

This work introduces online LLM-based AAD, a novel optimization paradigm that treats the algorithm itself as a state-dependent decision variable and proposes OnDesign, a multi-agent framework that reconciles competing design perspectives to synthesize executable algorithms and uses execution feedback to refine how runt...

Zhi-Yao Zhang, Yi-Chen Li, Xing-Yu Wu et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.