Code generation has become a core capability of large language models (LLMs), and code data is central to the pre-training process that develops it. As obtaining public code of sufficient quality becomes more difficult, further progress cannot rely on corpus expansion alone. Tiered data management instead applies incre...
Cheng-Ying Tu, Heng-Yu Zhao, Shuai-Kang Xue et al.· 0 citations
Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, selecting rows from a fixed pool or calling a generator, whereas a human data engin...
Bing-Xiang He, Peng Sun, Shi-Hao Peng et al.· 0 citations
This work proposes NOSA, a trainable sparse attention mechanism natively designed for KV cache offloading that explicitly constrains the volume of CPU-GPU KV transfers, thereby achieving low communication overhead and high decoding throughput.
Yu-Xiang Huang, Peng-Jie Wang, Ji-Cheng Han et al.· arXiv.org· 4 citations· ⚡1
UltraX is proposed, a function-calling refinement framework for large-scale pre-training data that completes the editing function space by introducing insertion in addition to deletion and modification, enabling fine-grained instance-level editing.
Xinlong Zhao, Dongsheng Liu, Hengyu Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.