Can LLM-Generated Backend Services Meet Performance Slos? A Controlled Study of Explicit Requirements and Measured-Feedback Repair

Abstract

Production backend services must satisfy functional contracts and operational objectives simultaneously, yet most evaluations of code-generating language models still emphasize compilation, unit-test success, or repository issue resolution. This study asks whether a coding-agent workflow can generate small backend services that satisfy explicit service-level objectives (SLOs), and whether measured runtime feedback changes the result. We built SLOBench, a reproducible local harness that combines versioned prompts, deterministic functional validation, process-isolated FastAPI execution, concurrent HTTP load, CPU and memory sampling, provenance hashes, and implementation-level analysis. The controlled pilot evaluated three services, metadata lookup, a bounded cache-backed API, and concurrent aggregation, under three conditions: functional-only prompting, functional prompting plus an explicit SLO, and one-step repair after measured feedback. Five generation attempts per task produced 45 implementations; each implementation was measured three times, yielding 135 final runs. All implementations passed the functional oracle and every final load run had zero observed request errors. Adding SLO language alone reduced P95 latency by a median 2.2% across 15 paired generation-task comparisons (bootstrap 95% interval -2.7% to 10.7%; 9/15 improved). Measured-feedback repair improved all 15 pairs, with a median 16.8% reduction relative to the SLO-prompted version (95% interval 15.7% to 20.2%). The two strictest latency targets were nevertheless never reached. Source review further showed that all repaired implementations bypassed portions of the ordinary framework path and three hard-coded the disclosed hot request. Measured feedback therefore improved performance on the declared workload consistently, but the evidence does not establish production readiness. The findings motivate hidden acceptance workloads, calibrated baselines, stronger post-repair contract testing, and multi-model replication in performance-oriented coding-agent evaluation.