Adapting Code LLMs to Domain-specific Languages via Synthethic Instruction Generation and Reinforcement Learning with AI Feedback in Low-resource Settings

Abstract

Large Language Models (LLMs) have significantly advanced automatic code generation in mainstream, high-resource, general-purpose programming languages. These models are trained on large instruction-code datasets with executable test environments, which can benchmark the performance of the outputs. However, they typically underperform on low-resource, proprietary Domain Specific Langauges (DSLs) due to limited training data. In this work, we introduce a fully automated, two-stage training pipeline for adapting coding LLMs to low-resource and proprietary DSL scenarios. Stage~1 performs supervised fine-tuning (SFT) using multi granular instructions synthesized directly from raw code and comments. Stage~2 conducts Reinforcement Learning with AI Feedback (RLAIF) by generating new instructions, and subsequently Direct Preference Optimization (DPO) by choosing a preferred output. We evaluate our approach using parser success, similarity and fidelity metrics, complemented with walkthrough studies. Our results suggest that this fully automated, two-stage pipeline can enable effective adaptation of LLMs to low-resource and proprietary DSL settings under realistic data and evaluation constraints.