Runtime efficiency is a critical factor that impacts both software quality and user satisfaction. There are many approaches proposed for code optimization to improve runtime efficiency. Traditional code optimization methods operate on intermediate representations (IRs) during compilation for static languages. They are effective but struggle to handle dynamic languages that do not require compilation. Recently, large language models (LLMs) have been leveraged to directly optimize source code in dynamic languages. However, these methods fail to identify suitable optimization targets and usually conduct incomprehensive single-level optimization. To address these challenges, we propose Optimo, a multi-level LLM-based code optimization approach built on a novel Mixture-of-Prompts (MoP) architecture. In the MoP architecture, Optimo identifies time-critical code structures as performance bottlenecks via differential profiling. These structures are then routed to some optimization strategies, akin to expert models in MoE, each tailored to optimize specific code patterns. Unlike traditional approaches that focus only on statement-level optimizations, Optimo operates at four levels of abstraction, ranging from coarse-grained algorithmic improvements to fine-grained optimizations in API usage. We evaluate Optimo on two code efficiency benchmarks, COFFE and Effibench. Our results demonstrate that Optimo achieves an up to 57.48% opt%, i.e., the percentage of optimized programs that are correct and at least 10% faster than the original programs, and an up to 3.97x speedup when optimizing human-written code, and it consistently outperforms the best baseline by up to 96.51% in terms of opt%. Furthermore, Optimo achieves an up to 42.42% opt% and an up to 13.51x speedup when optimizing LLM-generated code.
Security patch detection (SPD) is crucial for maintaining software security, as unpatched vulnerabilities can lead to severe security risks. In recent years, learning-based SPD approaches have achieved promising results on source code. However, they are generally not directly applicable to closed-source and proprietary software, which constitute a substantial portion of real-world systems, because such software typically releases patches only in binary form while its source code remains inaccessible. Despite the impressive performance of code large language models (LLMs) on code intelligence and binary analysis tasks, such as decompilation and compiler optimization, their potential for detecting binary security patches remains largely unexplored, revealing a notable gap between their demonstrated low-level code understanding capabilities and this security-critical task. To narrow this gap, we construct a large-scale binary patch dataset comprising 19,448 samples with two representations, assembly code and pseudo-code, and systematically evaluate 19 code LLMs of varying scales for binary SPD under both prompting and fine-tuning settings. Our initial study demonstrates that directly prompting off-the-shelf code LLMs remains ineffective; even advanced prompting strategies cannot compensate for the lack of task-specific knowledge. In contrast, fine-tuning proves highly effective, with pseudo-code representation consistently yielding the best performance. The best fine-tuned model, LLM4Decompile-9B-v2, achieves 0.915 in accuracy, 0.897 in F1 score, and 0.058 in false positive rate on pseudo-code. On average, models fine-tuned on pseudo-code improve accuracy by 27.5% and F1 score by 46.1%, while reducing the false positive rate by 46.2%, relative to their assembly-based counterparts. To understand this advantage, we analyze pseudo-code and assembly code from the perspectives of embedding-space similarity and code naturalness, and find that pseudo-code more closely resembles source code, making it better aligned with the source-code-centric pretraining of code LLMs. Motivated by this finding, we further augment the pseudo-code dataset with source code data, yielding additional gains after fine-tuning, particularly for smaller models. Beyond in-distribution evaluation, we further assess representative fine-tuned code LLMs under deployment-relevant distribution shifts and find that their robustness varies across shift types. Model performance remains stable under decompiler shift and is relatively preserved under architecture shift, suggesting that fine-tuned code LLMs retain meaningful robustness under practical distribution shifts in binary security patch detection.
Qingyuan Li, Binchang Li, Cuiyun Gao et al.· ACM Transactions on Software...· 3 citations· ⚡1