CoGate is proposed, a confidence-gated co-decoding approach that controls the expert's influence on the co-decoding process based on its confidence, and outperforms existing co-decoding methods (CoSec+) across multiple benchmarks.
Abstract
Large language models are widely used for code generation, but they can also produce insecure programs due to patterns learned from their pretraining data. Decoding-time steering has become an important solution to this problem: a small expert model is combined with the target model at each step to generate more secure code, which is referred to as co-decoding. However, the acceptance rule for existing co-decoding approaches does not consider the expert model's confidence. When the security expert is unconfident due to unseen patterns or out-of-distribution (OOD) contexts, its guidance can therefore be misleading. To address the challenge, we propose CoGate, a confidence-gated co-decoding approach that controls the expert's influence on the co-decoding process based on its confidence. We implement our approach and evaluate it across multiple LLM backends (CodeGen, DeepSeek-Coder, Qwen-Coder, StarCoder) on several code generation benchmarks (HumanEval, security suite, and CWEval). Our approach outperforms existing co-decoding methods (CoSec+) across multiple benchmarks, achieving up to a 12.6% gain of Func-Sec@10 on CWEval.
Six state-of-the-art quantization methods are evaluated on two representative large code model families using the multilingual McEval and CoderEval benchmarks for Python and Java to provide practical guidance for selecting quantization strategies for deploying large code models on resource-constrained hardware.
Saima Afrin, MD Zahidul Haque, A. Mastropaolo· 0 citations
A novel method is developed, SecVecCoder, leveraging task vectors to produce trustworthy code that is simultaneously functional and secure without the need for post-generation adjustment, and achieves a decoding latency within 0.6% of the base model's, on average.
F. Wang, Anudeep Das, M. Nagappan et al.· 0 citations
The first systematic study of model editing as a model-level hardening mechanism for secure code generation is conducted, evaluating 3 state-of-the-art editing methods across diverse LLM families and comparing them with CoSec, a representative inference-time approach, focusing on security, robustness, generalization, and functional correctness.
Weifeng Sun, Quanjun Zhang, Yuchen Chen et al.· 0 citations
It is confirmed that memorization persists in modern LLMs and is influenced more by a complex interplay of training domain, dataset composition, architectural choices, and content characteristics, rather than parameter count alone.
Xiaoyu Cheng, Kundi Yao, Pengyu Nie et al.· AIware· 0 citations
Large language models for code generation often produce incorrect solutions without reliable indicators of failure. We study whether uncertainty estimation methods developed for natural language transfer to code generation, and whether such signals can improve code generation via selective self-correction. We evaluate five uncertainty methods: mean token entropy, verbalized confidence, $P(\text{True})$, entropy ensembles, and semantic entropy probes, across three small code LLMs on HumanEval and BigCodeBench. We find that multi-sample $P(\text{True})$ achieves the strongest correlation with correctness, while all the other methods, including semantic entropy probes, yield only weak correlation. We then use these uncertainty signals to drive three self-correction policies: adaptive decoding, uncertainty-based regeneration, and verification-based regeneration. Our results reveal a stronger negative finding than anticipated: uncertainty-based self-correction fails to reliably improve Pass@1, degrading accuracy in 5 of 6 configurations across both benchmarks ($-3$pp to $-10$pp), and adaptive decoding degrades accuracy in 4 of 6 configurations. Only verification-based self-correction reliably improves Pass@1, with gains of $+6$ to $+26$ percentage points on HumanEval and $+8$ to $+20$ percentage points on BigCodeBench, scaling inversely with baseline strength. These findings replicate consistently across both benchmarks and suggest that cheap uncertainty estimators are insufficient on their own to improve code correctness, and that their practical value lies in serving as gating signals for costlier execution-based correction loops rather than as standalone substitutes for verification.
Pranav Rakasi, Maanas Lalwani, Arnav Srivastava et al.· 0 citations
Numerous tools have been developed to uncover secrets and credentials in source code. However, they often rely on predefined rules that yield a high rate of false positives. We conduct an in-depth investigation into the performance of different AI models in secret detection. In non-generative models, we found that the choice of feature set greatly influences the performance in secret detection. These models demonstrated the best performance using the context feature, i.e., code surrounding the secret value. Particularly, CodeBERT outperforms other models with an MCC and F1-scores of 88% and 89%, respectively. When we applied the models to an unseen dataset of secrets, the top models were Random Forest and CodeBERT, achieving an MCC score of around 65%. In generative models, we observed moderate performance (a maximum MCC of 63%) and a high number of false positives. The investigation of incorrect predictions revealed that many are within test-path files and that LLMs tend to be cautious, flagging potential secrets to promote secure coding practices. We noted that prompt engineering reduces false predictions significantly, yet human intervention remains necessary.
Farnaz Soltaniani, Mohammad Ghafari· Empirical Software Engineeri...· 1 citation