Skip to content
Open access

WHEN CORRECT CODE IS NOT ENOUGH: ASSESSING THE PRODUCTION READINESS OF LLMGENERATED SOFTWARE

Nov 2026 · International Journal of Software Engineering & Applications · 0 citations · 24 references

Abstract

Large language models (LLMs) increasingly generate executable software, but functional correctness alone does not establish production readiness. This study evaluates GPT-5.6 Sol, Terra, and Luna across 15 algorithmic and production-oriented tasks using five generations per model-task pair, yielding 225 generated programs plus 15 fixed references. The layered testbed combines nominal and hidden-edge-case checks, moderate and severe production-like stress, metamorphic validation, reference-normalized runtime and memory, structural quality, static analysis, implementation-strategy inspection, and generation cost. All 800 functional and 368 applicable robustness checks passed, while production-stress success fell to 95.16% for Sol, 95.60% for Terra, and 93.38% for Luna. Cross-model stress differences were not significant, but generation cost differed strongly. A Quick Sort case showed more than a 500x runtime spread among behaviorally correct implementations. The results show that production-oriented evaluation must extend beyond pass/fail correctness to include stress behavior, resources, implementation strategy, and cost

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.