Skip to content
Book Open access

Diagnosing and Correcting Geometric Bias in Diversity-Driven Code Instruction Data Selection

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · pp. 6219-6230 · 0 citations · 41 references

TL;DR

Semantically Anchored Geometric Coverage (SAGC), a framework for reshaping representation geometry using LLM-generated macro-intent labels as semantic anchors for dense micro-context representations, is proposed, using macro-intent anchors to constrain dense-space exploration toward high-level algorithmic structure.

Abstract

Diversity-driven data selection offers a scalable way to reduce code instruction-tuning costs, yet it relies on a fragile assumption: geometric spread in representation space should correspond to training-useful semantic coverage. In code data selection, our diagnosis reveals a geometric bias behind this assumption: dense embedding geometry can be strongly influenced by rare surface forms, verbose constraints, and implementation artifacts, causing diversity objectives to over-explore locally sparse or globally peripheral samples whose training utility is not guaranteed. To correct this bias, we propose Semantically Anchored Geometric Coverage (SAGC), a framework for reshaping representation geometry using LLM-generated macro-intent labels as semantic anchors for dense micro-context representations. Rather than replacing standard diversity objectives, SAGC reshapes the geometry exposed to them, using macro-intent anchors to constrain dense-space exploration toward high-level algorithmic structure without requiring downstream validation signals, gradient feedback, or evaluation results. Using only 20k samples selected from a 446k pool, SAGC reaches 0.4006 on LiveCodeBench, approaching full-data training at 0.4027. Further diagnostics and out-of-benchmark evaluations show that semantic anchoring provides a structured correction to this geometric bias.

Read PDF

Similar papers

#artificial intelligence Preprint Nov 2026

GraphScent: Calibrating Directional Relevance Signals for Bounded-Context Code Localization

Running open-weight language models locally on a single consumer GPU requires replacing expanding multi-turn exploration histories with short, stateless code inspections. Stateless repository localization, however, hinges on a foundational question: when a frozen language model evaluates an unedited code entity from a...

R. A. Naagarajan · 0 citations
Book Open access 2027

Selecting Memory Layouts for Model-Based Microarchitectural Performance Evaluation

Virtual memory research increasingly relies on empirical performance models that use simulated address translation metrics (x)---such as TLB miss rate and page-walk cycles---to predict benchmark runtimes (y) for new hardware designs. The accuracy of these models depends on the training data: a set of (x,y) pairs collec...

Mohammad Agbarya, Idan Yaniv, Dan Tsafrir · 0 citations
Open access Nov 2026

Machine Learning-Guided Design Space Exploration for Instruction-Level Approximate Computing

Approximate Computing (AC) trades reduced precision for energy savings in error-tolerant scenarios, but identifying the right instruction-level placement demands extensive, computationally prohibitive Design Space Exploration (DSE), further burdened by costly simulation-based evaluation. To address this, we propose a h...

Gregório K. Neto, Marco Antônio de R. Zárate, G. G. Damasceno et al. · 0 citations

Hardware-Aware Features for CUTLASS Kernel Selection

A hardware-aware representation for CUTLASS kernel selection is introduced that augments candidate configurations with statically computable estimates of induced hardware behavior, showing that explicitly representing candidate-induced hardware behavior provides a useful inductive bias for learned kernel selection.

Shriram Chandran, Dominic Rinderer, Yakup Budanaz et al. · 0 citations
Book Open access Oct 2026

AlphaCu: A Transformation-Driven Synthesis Framework for LLM-Based GPU Kernel Generation

Large language models show promise for code generation, yet their effectiveness in producing high-performance GPU kernels remains insufficiently understood. Recent agentic workflows improve over direct prompting through specialized generation roles and execution feedback, but our characterization of a representative wo...

Yang-Jie Zhou, Xing-Chen Ma, Zi-Han Liu et al. · 0 citations

ReMCTS: Reflection-Enhanced Monte Carlo Tree Search for Code Generation

ReMCTS, an execution-grounded, memory-augmented, LLM-guided MCTS-style search framework that organizes program candidates as tree states, retains branch-local debugging context, retrieves failure experience across branches, and distinguishes failed checks from unavailable evidence is presented.

Hui-Fei Wang, Xin-Yi Huang, Yi-Heng Sun et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.