Nov 2026· Proceedings of the 35th ACM International Conference on Information and Knowledge Management· pp. 6219-6230· 0 citations· 4 references
Abstract
Diversity-driven data selection offers a scalable way to reduce code instruction-tuning costs, yet it relies on a fragile assumption: geometric spread in representation space should correspond to training-useful semantic coverage. In code data selection, our diagnosis reveals a geometric bias behind this assumption: dense embedding geometry can be strongly influenced by rare surface forms, verbose constraints, and implementation artifacts, causing diversity objectives to over-explore locally sparse or globally peripheral samples whose training utility is not guaranteed. To correct this bias, we propose Semantically Anchored Geometric Coverage (SAGC), a framework for reshaping representation geometry using LLM-generated macro-intent labels as semantic anchors for dense micro-context representations. Rather than replacing standard diversity objectives, SAGC reshapes the geometry exposed to them, using macro-intent anchors to constrain dense-space exploration toward high-level algorithmic structure without requiring downstream validation signals, gradient feedback, or evaluation results. Using only 20k samples selected from a 446k pool, SAGC reaches 0.4006 on LiveCodeBench, approaching full-data training at 0.4027. Further diagnostics and out-of-benchmark evaluations show that semantic anchoring provides a structured correction to this geometric bias.
VeriSelector is proposed, the first data selection framework for Verilog code generation that jointly optimizes quality and diversity, and outperforms full-dataset training and state-of-the-art baselines.
Yi-Heng Shen, Wei Zheng, Xiao Wei et al.· Information Sciences· 0 citations
Virtual memory research increasingly relies on empirical performance models that use simulated address translation metrics (x)---such as TLB miss rate and page-walk cycles---to predict benchmark runtimes (y) for new hardware designs. The accuracy of these models depends on the training data: a set of (x,y) pairs collec...
Mohammad Agbarya, Idan Yaniv, Dan Tsafrir· Proceedings of the 32nd ACM...· 0 citations
A hardware-aware representation for CUTLASS kernel selection is introduced that augments candidate configurations with statically computable estimates of induced hardware behavior, showing that explicitly representing candidate-induced hardware behavior provides a useful inductive bias for learned kernel selection.
Code generation has become a core capability of large language models (LLMs), and code data is central to the pre-training process that develops it. As obtaining public code of sufficient quality becomes more difficult, further progress cannot rely on corpus expansion alone. Tiered data management instead applies incre...
Cheng-Ying Tu, Heng-Yu Zhao, Shuai-Kang Xue et al.· 0 citations
Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful. Because this safety-degrading data looks benign, standard content filters struggle to catch it. We tackle this problem by developing SimpleLoRIF, a scalable gradient embedding method...
S. Reddy, Amro Abbas, Anshuman Suri et al.· 0 citations
Large Language Models (LLMs) have significantly advanced automated code generation, yet they frequently produce code with critical security vulnerabilities. While recent alignment techniques (e.g., supervised fine-tuning and preference optimization) have attempted to mitigate this, existing methods often suffer from th...
Meng Tian, Youliang Yuan, Zhiqing Zhong et al.· Proceedings of the 41st IEEE...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.