This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts and analytically derives a minimal explicit deployment specification for information that cannot be reliably inferred from repository artifacts.
Abstract
Large Language Models (LLMs) are increasingly used to generate executable software environments from repository artifacts. However, functional executability does not necessarily imply conformity with architectural, security, workflow, and production intent. This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts. Three heterogeneous repositories combining Python, Node.js, .NET, React, Rust, Java, Redis, PostgreSQL, and MySQL-compatible infrastructure were evaluated using deterministic end-to-end HTTP oracles and manual structural comparison. All three generated environments became functionally operational, although one required a Rust base-image update from version 1.85 to 1.88. The model correctly reconstructed service topology, application ports, infrastructure dependencies, service hostnames, a background worker, hidden proxy configurations, and a file-based Docker secrets mechanism. However, it consistently omitted network segmentation, multi-stage builds, dependency-layer caching, live-reload volumes, production frontend serving, restrictive backend-port policies, and cross-platform build logic. Based on these observations, the study formalizes the distinction between functional correctness and deployment-intent fidelity and analytically derives a minimal explicit deployment specification for information that cannot be reliably inferred from repository artifacts.
Repository-level code translation is critical for modernizing legacy systems, yet existing approaches based on large language models (LLMs) operate at the file level and fail to scale to codebases with complex inter-file dependencies. This limitation is evident in our industrial setting, where we aim to migrate a production repository (STAR) from Java to Kotlin, but file-level approaches produce fragmented results and fail to achieve end-to-end correctness. In this paper, we show that the primary cause of failure at the repository level is dependency inconsistency. Through an empirical study on open-source and industrial systems, we find that most errors arise from unresolved cross-file dependencies that cannot be effectively addressed by iterative feedback alone. We propose a dependency-aware incremental migration framework that elevates the unit of translation from individual files to dependency-consistent batches. Our approach constructs a dependency graph, groups interdependent files, and performs batched translation with iterative compile- and test-driven validation. We evaluate our method on a 51K line of code (LOC) industrial system and multiple repositories across interoperable language pairs (Java-Kotlin, Java-Scala, and C#-F#). On the STAR repository, file-level approaches achieve 38.16% compilation and 9.39% test success, whereas our approach achieves 100% compilation and test success across the evaluated settings, converging within a small number of iterations. These results show that dependency-aware batching improves scalability and reliability in repository-level code translation.
Sivajeet Chand, Alexander Pretschner, Steve Haupt et al.· 1 citation
Large Language Models (LLMs) are now widely used for code generation, yet even syntactically correct output may contain logical and semantic errors that remain invisible until runtime, particularly in framework-driven applications, where correctness depends on dependency injection, framework conventions, configuration, library compatibility, and database interaction. This paper presents GenTest, an open-source platform for context-aware dynamic generation of Java code using LLMs, runtime compilation and class loading, Spring bean registration, JPA/PostgreSQL execution, and declarative YAML-based test validation. GenTest supports multiple LLM providers through a provider-agnostic interface and evaluates generated components within a live Spring Boot application context. Across 209 test cases and 933 assertions, GenTest achieves a 63.9% assertion pass rate, and 94.2% of failures occur after successful compilation, confirming that execution-centered evaluation reveals failure modes invisible to static analysis.
Muhammed Suphi Şeyhkuruş, Tolga Ovatman· Annual International Compute...· 0 citations
: This research focuses on Maven configuration and structure updates and explores how the CodeT5 model can automate the migration of Java 8 projects to Java 17. Manually migrating large Java applications is a tedious and error-prone task that requires significant developer time and attention. To create a high-quality training dataset, Java 8 projects are collected from GitHub and manually converted to Java 17 using large-scale modern language models such as ChatGPT and Claude. Each repository is migrated module-by-module to ensure structural and functional accuracy. This process has created a parallel dataset of approximately 100 repositories, formatted as a JSONL file to allow efficient fine-tuning of small CodeT5 models. The model fine-tuning is evaluated using standard metrics, including BLEU, ROUGE, and BERTScore. The results show strong semantic agreement between the generated and reference outputs, with a BERTScore F1 of 0.982 and token precision of 0.9990. The model accelerates parts of migration but remains unsuitable for fully automated use due to truncated outputs and incomplete Java file structures, with an average output-length ratio of 0.8 and an accuracy of 62% for the Java version.
Ayush Luhar, Dev Trivedi, Vatsal B. Patel et al.· Proceedings of the 21st Inte...· 0 citations
Configuration management in large-scale cloud networks is increasingly challenging due to vendor heterogeneity, diverse configuration interfaces, and rapid configuration evolution across the network life cycle. Existing approaches rely heavily on vendor- and interface-specific templates and scripts, which are difficult to validate, costly to maintain, and scale poorly. We introduce AliYANG, a YANG-based configuration modeling framework that unifies configuration representation across vendors and management interfaces. It extends YANG to capture CLI semantics and derives a vendor-agnostic core model that separates configuration semantics from vendor-specific implementations. We further present NetCMDB, the production software infrastructure for AliYANG, which compiles models into typed configuration objects and supports end-to-end, model-driven configuration workflows. As networks evolve, manually constructing and maintaining models becomes a bottleneck. We incorporate LLM-assisted automation to facilitate vendor model augmentation, core model design, and bidirectional translation code generation. We report our three-year production deployment experience managing hundreds of thousands of devices, present evaluation results and case studies, and share lessons from operating a model-driven configuration system at cloud scale.
Mohan Yu, Xumiao Zhang, Zhecheng An et al.· 0 citations
The results show that specification size alone does not predict implementation quality and that cross-agent transfer can produce substantial agent-dependent degradation, and suggest that specifications in heterogeneous SDD workflows should not automatically be treated as agent-neutral artifacts.
Oleg Grynets, O. Ilchuk, Dariia Zatulna et al.· 0 citations
Enterprise compliance management requires rapid adaptation to evolving regulatory frameworks (e.g., DORA, AI RMF, FedRAMP) and tight remediation SLAs. Traditional static orchestrators often fail in hybrid cloud environments where event-driven assessments demand that automation code adapt to runtime context in seconds. This paper presents lessons learned from evaluating six large language models for AI-driven workflow generation in a production enterprise platform, benchmarked across 29 real-world IT automation scenarios, two generation pipeline architectures, and eight independent runs per prompt-model-pipeline configuration (2,784 runs total). Our initial pipeline used monolithic workflow generation, achieving 31.5-82.8% structural success rates (JSON schema validity and correct UI rendering), with most models struggling on complex JSON generation. We developed a redesigned piecewise pipeline that decomposes workflow construction into variable scaffolding, base block assembly, and nested block generation, raising structural success to 74.1-97.8% across all models. We analyze production tradeoffs including cost (USD 0.008-0.20 per workflow), latency (under 50s for interactive use), and model selection. Piecewise decomposition enables smaller models (e.g., mistral-small at 95.7% structural success and USD 0.01 per workflow) to reach production viability, removing dependency on expensive frontier models. While mistral-medium-2505 and gpt-oss-120b achieved the highest structural success (96.1% and 97.8%), mistral-medium-2505 carries a 19x cost premium versus mistral-small. Our deployment lessons highlight the need to separate structural validity from semantic correctness (logical fulfillment of user intent) and provide a solution for model-agnostic, scalable automation in cloud engineering.
Xavier Wrenn, Radoslav Raykov, Aleksandar Angelov et al.· 0 citations