Skip to content

CoSTAR: Data Synthesis-Driven Constraint-Aware COBOL Section Summarization for Legacy System Modernization

Sep 2026 · 0 citations · 53 references
Computer Science

TL;DR

Results show that CoSTAR enables small, locally deployable LLMs to achieve performance competitive with substantially larger LLMs for privacy-sensitive COBOL legacy systems.

Abstract

COBOL remains critical to governments, financial institutions, and large enterprises; yet, aging technologies, shrinking expertise, and missing documentation make modernization of COBOL-based legacy systems increasingly urgent. Before migration, code summarization is a common practice to support legacy system understanding. However, COBOL code summarization, especially on section-level, faces two key challenges: data scarcity and migration constraint preservation. To address these challenges, we propose CoSTAR, an integrated framework that combines execution-validated data synthesis with constraint-aware model training. CoSTAR repurposes general-purpose programming tasks to synthesize execution-validated COBOL code-summary data through LLM-based generation to overcome data scarcity. Based on the synthesized data, CoSTAR augments target sections with relevant data declarations and natural-language explanations, and uses constraint-guided structured rationales to train smaller base LLMs. The trained LLMs preserve the migration constraints for COBOL section summarization. We evaluate CoSTAR on both public and confidential enterprise COBOL systems. CoSTAR effectively synthesizes 3,764 execution-validated training instances. Based on these instances, CoSTAR built on 7B/8B base LLMs can improve these LLMs with average relative gains of 25.38% on ROUGE-L, 53.84% on METEOR, and 37.22% on chrF. In real-world enterprise evaluation, CoSTAR built on only Qwen3-8B, outperforms the enterprise-deployed Qwen3-235B in accuracy, completeness, and conciseness. These results show that CoSTAR enables small, locally deployable LLMs to achieve performance competitive with substantially larger LLMs for privacy-sensitive COBOL legacy systems.

View source

Similar papers

DatalogBench: Evaluating Large Language Models on Text-to-Datalog Synthesis

DatalogBench is presented, a benchmark of 136 text-to-Datalog synthesis tasks curated from existing Datalog-based artifacts that identifies recursive reasoning and decomposition as open challenges for current LLMs and agents, and offers a reliable, execution-grounded measure of both.

Yuan Li, Han-Yun Jiang, Guo-Wei Tian et al. · 0 citations
Preprint Aug 2026

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

StateTune is presented, which reformulates LLM-assisted EDA tuning as a closed-loop, state-carrying process that achieves the strongest final hypervolume on all six benchmark blocks, showing a stable improvement in frontier quality across the full matrix.

Kunlong Li, Shang-Shang Yao, Su Zheng et al. · 0 citations
#small language model Book Open access Sep 2026

Code Transformation Rule Synthesis using LLMs: Potential and Limits

A novel empirical study targeting three domain- specific languages for transformation rules, namely Comby, GritQL, and Ast-Grep, which shows great potential for LLMs to generate sound, correct, generalizable, and reusable rules.

Axel Allain, Aymeric Blot, D. Khelladi et al. · 1 citation
Preprint Sep 2026

Structure-augmented LLMs for High-Level Synthesis Pragma Optimization

In the agentic flow, the PRISM codegen outperforms other baselines when optimizing complex code and drives the average normalized improvement across the HLS-Eval suite to 26.4\%.

Hao-Cheng Xu, Ye Qiao, Phyo Pyae Moe Aung et al. · 0 citations

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

Dynamic Retrieval-based Policy Generation (DRPG) is proposed, a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement.

Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang et al. · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.