Skip to content
Preprint

Instruction Alignment for Binary Code Representation Learning

Aug 2026 · 1 citation · 76 references
Computer Science

TL;DR

The preliminary study reveals that models finetuned for function-level binary code similarity exhibit substantially better instruction alignment than their pre-trained model, suggesting a strong correlation between instruction alignment and function-level embedding quality.

Abstract

Binary code representation learning is a fundamental problem in software security and reverse engineering. Existing methods mainly learn function-level embeddings that capture coarse-grained semantic relationships between binary functions, but they largely ignore fine-grained instruction-level correspondences. This limitation misses valuable supervision signals available from compiler debug information, which can support the learning of more accurate and interpretable binary code representations. We propose to leverage instruction alignment knowledge to further improve binary code representation learning. Our preliminary study reveals that models finetuned for function-level binary code similarity exhibit substantially better instruction alignment than their pre-trained model, suggesting a strong correlation between instruction alignment and function-level embedding quality. Motivated by this observation, we design a training approach that explicitly incorporates instruction alignment as an auxiliary training objective. Our experiments show that instruction alignment training improves retrieval accuracy and provides more discriminative signal for the model's similarity judgments.

View source

Similar papers

#small language model Book Open access Aug 2026

Invariant Pretraining for Robust Code Representations

InvPT applies semantic-preserving transformations to the corpus and combines masked language modeling with multi-positive supervised contrastive learning that treats all augmentations of the same source function as positives, mixing self-contrast pairs with invariant-contrast pairs for positives of varying difficulty.

Yifeng He, Yun-Di Xu, Christopher Castro Gaw Gonzalo et al. · 0 citations
2026

Generative vs. Discriminative? An Empirical Study on Code Understanding Classification Tasks

Automated code understanding is crucial for software reliability and maintainability. Encoder-only pretrained models excel in code classification tasks such as vulnerability detection, cross-language clone detection, and exception classification due to their bidirectional context awareness. However, the dominant “pre-t...

Yu-Guo Liu, Yu-Hua Ma, Rong-Cun Wang et al. · 0 citations
Open access Sep 2026

DuaLoc: Dual-Encoder Bug Localization with Bug-Report-Conditioned Attention and Contrastive Learning

DuaLoc combines two pre-trained language models: UniXcoder for the semantic understanding of source code and GraphCodeBERT for awareness of data-flow structure and fine-tuned with a contrastive objective that shapes the embedding space around the localization task.

Amany AlBatlaa, M. Abdullah-Al-Wadud · 0 citations
Open access Oct 2026

Self-Adaptive Code Representation: Grammar-Guided or Token-Based?

Recent advances in code intelligence are largely driven by large language models (LLMs), which represent code as token sequences and achieve strong results on many software engineering tasks. However, standard token-based approaches often struggle to capture the inherent structural regularities of programming languages...

Qing-Lin Liang, Zhao Zhang, Chen Liu et al. · 1 citation
#artificial intelligence Preprint Sep 2026

Retrieval-Augmented Generation for Scientific Code Understanding

The results indicate that front-loading code understanding into a reusable, codebase-specialised vector store enables small local models to deliver grounded and repository-specific answers, making the agent well suited as a privacy-preserving development tool for in-house scientific codebases.

A. Nobile, Andreas Adelmann, Mohsen Sadr · 0 citations
Aug 2026

CodeLite: A Low-Cost Framework for Code Classification Tasks via Enhanced Code Representations and Lexical Fusion

This work proposes ‘CodeLite’, a relatively lightweight framework that integrates moderately sized code models, such as CodeBERT, with traditional frequency-based techniques, achieving better performance and computational efficiency compared to larger models.

Aman Swaraj, Sandeep Kumar · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.