Retrieval-Augmented Generation (RAG) has become a cornerstone in software engineering for enhancing Large Language Models (LLMs) with domain-specific knowledge. However, adapting retrievers to evolving code repositories remains challenging due to the noise and redundancy inherent in massive code corpora. Standard fine-...
Yuxuan Chen, Ming-Wei Liu, Guang-Sheng Ou et al.· 0 citations
Large language models (LLMs) perform remarkably well on complex tasks, yet remain highly vulnerable to prompt injection attacks, where malicious instructions embedded in external data can override user intent. Existing defenses remain limited by model fine-tuning requirements, vulnerability to adaptive attacks, or reli...
Lu-Man Zhao, Ming-Hui Xu, Yue Zhang et al.· 0 citations
Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based filtering has proven effective in selecting high-quality subsets from web-scale corpora, especially for high-resource languages, low-resource languages face challenges d...
Vinko Sabolcec, Bettina Messmer, Yassine Turki et al.· 0 citations
CLIP serves as a foundational vision-language model and the de facto vision encoder for downstream VLMs such as LLaVA. Post-training offers a lightweight route to refine CLIP, but recent work argues that the standard contrastive loss is unsuitable for post-training due to catastrophic forgetting under small batches, mo...
Zi-Dan Wang, Ya-Qian Li, Xiao-Kai Zhang et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
This work introduces MetaEncoder, which fine-tunes a pre-trained Muse-Glimmer 30B decoder into an instruction-following decision-making encoder, and employs a bi-encoder architecture trained via unidirectional contrastive learning for request-candidate alignment.
Jian-Peng Cheng, Guang-Yu Sun, Aashu Singh et al.· 0 citations
This work proposes envelope sampling, a theoretically-grounded method for judge recalibration that seeks to minimize an upper bound on the regret of the post-trained model under the assumption that the human reward and re-calibrated reward lie in an $L^2$ ball around the judge.
Sanjit Dandapanthula, Shuvom Sadhuka, Samir Khan et al.· 0 citations
Vertical-domain few-shot classification remains challenging for small language models, as limited supervision makes it difficult to acquire domain-specific decision knowledge. On-Policy Distillation (OPD) can improve teacher-guided adaptation by supervising student-generated rollouts, while GRPO-based reinforcement lea...
Heng Li, Yong Zhang, Ning Cheng et al.· 0 citations
Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent models (i.e., the pretrained models before fine-tuning). This trade-off is especially limiting for queries that require both specialized and...
Ke-Nan Tang, An-Dong Hua, Cheng-Xuan Qian et al.· 0 citations
Group favoritism is conformity to observed group behavior, carried to strangers by the label and overridden by individual reputation, and predicts responses to betrayal, scandal, and a free offer to change group.
Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale. We close t...
Marjan Celikik, A. Ramallo, Javier Morales· 0 citations
Understanding large language model (LLM) reasoning requires looking beyond behavioral performance to examine how reasoning ability is reflected in internal organization. Inspired by neuroscience findings linking higher intelligence to stronger small-world organization in functional brain networks, we investigate small-...
Zheng Huang, Sansheng Cao, En-Pei Zhang et al.· 0 citations
Deploying large language models in production often requires constraining outputs to structured formats such as JSON or XML, and prior work treats the resulting accuracy loss as an inherent `structure tax'. We re-examine this claim by evaluating a battery of models, datasets and schemas, measuring task accuracy, confid...
