OrthCoder: Balancing Security and Utility in Secure Code Generation via Orthogonal Adaptation
Abstract
Large Language Models (LLMs) have significantly advanced automated code generation, yet they frequently produce code with critical security vulnerabilities. While recent alignment techniques (e.g., supervised fine-tuning and preference optimization) have attempted to mitigate this, existing methods often suffer from the notorious safety tax—a severe degradation in general functional utility, particularly on complex programming tasks. This trade-off can arise from parameter interference, where security-driven updates interact with the model’s pre-trained functional representations. Alternative approaches that avoid modifying model parameters (e.g., inference-time methods) can alleviate this issue but often introduce additional inference overhead or instability. In this work, we focus on snippet-level secure code generation, where a model generates a self-contained function or short code routine from a natural-language prompt. Within this setting, we propose OrthCoder, a novel secure code generation framework based on subspace decoupling. OrthCoder employs a Dual-LoRA architecture, dedicating separate adapters to functional utility and security alignment. By introducing a geometric orthogonality constraint (Lorth) during joint training, we encourage the utility and security adapters to learn less-overlapping parameter updates. Extensive evaluations across four diverse foundational models (∼7B scale) on seven benchmarks covering snippet-level security and general code-generation utility demonstrate that OrthCoder achieves state-of-the-art performance on joint security-utility benchmarks such as CodeGuard+ and CWEval. More importantly, OrthCoder mitigates the severe safety tax observed in existing alignment baselines. Furthermore, analyses of the learned adapter parameters show substantially reduced overlap between utility and security updates, supporting the effectiveness of the proposed decoupling mechanism.